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INTELLIGENCE  PREPARATION  OF  THE  BATTLEFIELD: 
CRITIQUE  AND  RECOMMENDATIONS 


1 . 0  SUMMARY 


The  purpose  of  this  paper  is  to  show  how  the  Intelli¬ 
gence  Preparation  of  the  Battlefield  'IPB)  analysis  process 
outlined  in  Circulars  TC30-27  and  TC34-3  could  be  improved 
by  incorporating  decision-analytic  techniques  and  computer¬ 
ized  decisions  aids  to  improve  the  judgmental  process  inher¬ 
ent  within  IPB.  To  accomplish  this  purpose,  the  paper  is 
divided  into  two  major  sections. 

Section  2.0  describes  the  necessary  dependence  of  IPB 
on  human  judgment.  In  particular,  it  provides  a  conceptual 
overview  of  the  causal  inference  problem  inherent  in  intel¬ 
ligence  analysis,  and  identifies  the  different  types  of 
judgment  inherent  in  each  of  the  five  steps  in  the  IPB 
analysis  process  proposed  in  TC30-27  and  TC34-3.  The  gen¬ 
eral  conclusion  is  that  except  for  terrain  analysis  (step  3) 
and  weather  analysis  (step  4),  these  circulars  fail  to  tell 
intelligence  analysts  how  they  are  to  make  the  judgments 
necessary  to  implement  the  proposed  IPB  analysis  process. 

Section  3.0  describes  the  results  of  research  studying 
how  well  people  make  the  judgments  inherent  in  IPB.  The  re¬ 
search  strongly  suggests  that  people's  ability  to  make  these 
judgments  can  be  improved  substantially  by  training  them  in 
decision-analytic  techniques  and  by  giving  them  access  to 
computerized  judgment  aids.  Two  judgment  aids  incorporating 
decision-analytic  techniques  are  described  in  detail,  for 
they  are  designed  to  assist  Staff  Intelligence  Officers  (G2/ 
S2s)  in  looking  at  the  battlefield  from  the  perspective  of 
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the  enemy  commander.  Since  both  aids  provide  displays  de¬ 
signed  to  help  analysts  convey  the  reasons  for  their  judg¬ 
ments  to  tactical  commanders,  they  provide  an  adjunct  to  the 
graphic  templating  process  in  IPB. 

Due  to  the  length  of  this  report,  the  reader  with 
limited  time  is  referred  to  Section  1.0,  which  provides  a 
detailed  overview  and  summary.  The  reader  who  is  only  in¬ 
terested  in  the  judgment-analytic  critique  of  IPB  (i.e., 
Section  2.0)  is  referred  to  Adelman,  Donnell,  and  Phelps 
(1980) . 

1 . 1  IPB's  Dependence  on  Judgment 

The  first  section  below  describes  the  limitations  in 
Circulars  TC30-27  and  TC34-3  regarding  the  necessary  judgments 
in  the  following  three  steps  of  the  proposed  IPB  analysis 
process:  mission  and  threat  evaluation  (step  1),  tactical 

intelligence  zone  determination  and  evaluation  (step  2) ,  and 
threat  integration  (step  5) .  The  second  section  describes 
the  different  types  of  judgments  involved  in  revising  IPB 
templates,  an  area  only  minimally  addressed  in  Circulars 
TC30-27  and  TC34-3. 

1.1.1  Basic  analysis  steps  -  The  purpose  of  mission 
and  threat  evaluation  (step  1)  is  to  review  order  of  battle 
holdings  so  as  to  identify  the  gaps  in  collected  intelli¬ 
gence  data,  and  thereby,  guide  future  collection  efforts. 
Bowen,  et  al.  (1975)  point  out  the  judgmental  nature  of 
order  of  battle  holdings.  The  intelligence  team  must  evalu¬ 
ate  a  large  amount  of  incoming  data,  much  of  which  may  be 
unreliable,  in  order  to  decide  whether  it  "knows"  the  order 
of  battle  holdings.  In  addition,  they  must  integrate  the 
order  of  battle  holdings,  complete  or  not,  into  a  judgment 
of  enemy  intent.  Although  the  proposed  IPB  process  uses 
templates  to  represent  the  output  of  this  judgmental  process, 


it  provides  no  means  for  representing,  pictorially  or  other¬ 
wise,  the  bases  for  the  judgments  underlying  threat  evalua¬ 
tion.  Furthermore,  TC30-27  and  TC34-3  fail  to  provide  an 
explicit  representation  of  the  probable  relations  between 
actual  enemy  intent  and  the  order  of  battle  holdings  that 
repesent  indicators  of  intent;  consequently,  it  offers  no 
explicit  rules  or  heuristics  describing  how  order  of  battle 
holdings  are  to  be  combined  into  judgments  of  enemy  intent. 
Scientific  research  on  causal  inference  strongly  suggests 
that  threat  evaluation  judgments  could  be  improved  by  pro¬ 
viding  general  information  about  probable  cause-effect  rela¬ 
tions  . 

Tactical  intelligence  zone  determination  and 
evaluation  (step  2)  is  supposed  to  identify  hov  the  opposing 
forces  ( OPFOR)  threat  evaluated  in  step  1  should  look  in  the 
battlefield,  in  general.  Circulars  TC30-27  and  TC34-3  ac¬ 
complish  this,  however,  only  to  a  limited  degree.  There  is, 
for  example,  no  discussion  of  the  general  indicators  for 
different  OPFOR  threats.  Yet  research  by  Johnson,  Spooner, 
and  Jaarsma  (1977)  suggests  that  this  would  be  a  valuable 
addition  for  tactical  commanders.  They  found  that  a  sample 
of  forty-three  captains  in  the  Intelligence  Officers  Advanced 
Course  knew  only  nineteen  of  the  forty-nine  separate  indica¬ 
tors  listed  in  Field  Manual  30-5,  Combat  Intelligence,  for 
four  of  the  OPFOR  threats  that  are  to  be  evaluated  in  step  1. 
TI  zone  determination  and  evaluation  could  be  improved  by 
indicating  (1)  the  relative  accuracy  of  individual  indica¬ 
tors  for  different  OPFOR  courses  of  action,  and  (2)  how  to 
combine  individual  indicators  into  a  global  assessment  of 
threat . 

The  objective  of  the  fifth  and  final  step  in  the 
proposed  IPB  analysis  process  is  to  relate  "how  the  enemy 
would  like  to  fight"  to  a  specific  terrain  and  weather  sce¬ 
nario  as  the  basis  for  determining  "how  the  enemy  might  have 


to  fight"  (TC34-3,  p.  5-1).  According  to  both  circulars, 
this  is  to  be  accomplished  through  the  use  of  situation, 
event,  and  decision  support  templates.  These  templates  are 
proposed  as  a  means  of  helping  commanders  "visualize"  enemy 
capabilities  in  a  particular  combat  setting.  They  require 
intelligence  analysts  to  make  a  number  of  different  kinds  of 
judgments.  Both  circulars  fail,  however,  to  tell  intelli¬ 
gence  analysts  how  to  make  these  judgments.  These  different 
judgments  are  identified  below  for  each  template,  in  turn. 

In  addition,  limitations  in  the  circulars  are  identified, 
and  suggestions  are  offered  for  their  improvement. 

The  situation  template  shows  how  enemy  forces 
probably  would  look  within  the  different  mobility  corridors 
under  consideration.  Underlying  the  development  of  this 
template  is  a  complex  judgmental  process.  We  quote  TC30-27 
on  this  point. 

"While  the  enemy  commander  may  not  have  unlimited 
options  as  to  possible  courses  of  action,  he  will 
probably  have  enough  options  to  make  the  analyst's 
job  of  determining  probable  courses  of  action  ex¬ 
tremely  difficult.  Situation  templates  are  derived 
based  on  the  best  military  judgment  of  the  analyst" 

(p.  4-4). 

TC30-27  and  TC34-3,  however,  provide  only  minimal  informa¬ 
tion  on  how  analysts  are  to  exercise  their  "...  best  mili¬ 
tary  judgment  ..."  when  developing  the  situation  templates. 
If  the  situation  template  is  to  reflect  military  judgment 
about  different  OPFOR  courses  of  action,  it  is  not  enough  to 
show  analysts  how  to  perform  a  terrain  and  weather  analysis. 
In  addition,  the  circulars  should  identify  the  factors  that 
are  to  be  considered  when  making  this  judgment,  for  the 
analysis  team  will  have  to  incorporate  judgments  about 
other,  more  ambiguous  factors,  such  as  perceived  U.S.  force 
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strength  and  risk,  that  an  OPFOR  commander  would  certainly 
consider  when  selecting  a  course  of  action.  Furthermore, 
the  circulars  should  discuss  the  trade-offs  that  OPFOR  com¬ 
manders  are  likely  to  make  when  evaluating  the  utility  of 
different  courses  of  action.  Rarely  will  it  be  true  that 
one  course  of  action  is  better  than  all  others  on  every  fac¬ 
tor.  Enemy  commanders  will  be  forced  to  differentially 
weight  aspects  of  their  doctrine  with  the  characteristics  of 
the  situation  immediately  facing  them.  TC30-27,  TC34-3,  or 
supporting  documentation  should  indicate  what  these  trade¬ 
offs  are  likely  to  be  under  different  terrain  and  weather 
constraints,  if  the  situation  template  is  to  accurately  rep¬ 
resent  military  judgment  under  different  circumstances. 

"The  event  template  is  a  time  and  logic  sequence 
of  enemy  tactical  indicators  or  events  which  are  keyed  to  a 
series  of  situational  templates"  (TC30-27,  pp.  1-6).  Although 
TC30-27  and  TC34-3  provide  no  example  of  an  event  template, 
it  appears  that  the  template  must  identify  the  different 
types  of  information  necessary  to  confirm  the  adoption  of  a 
particular  course  of  action.  Such  information  is,  in  fact, 
required  to  complete  the  "events  analysis  matrix"  (see 
TC34-3 ,  p.  5-7) .  To  complete  this  matrix,  analysts  must  in¬ 
dicate  those  "event  activities"  they  expect  to  see,  as  well 
as  when  and  where  they  expect  to  see  them,  for  each  avenue 
of  approach.  In  doing  this,  the  analysts  are  essentially 
making  a  series  of  conditional  probability  judgments.  That 
is,  they  must  say  something  like,  "If  the  enemy  actually 
took  this  particular  course  of  action,  then  these  indicators 
and  events  have  a  higher  probability  of  being  observed  than 
others."  The  word  "probability"  must  be  emphasized  because 
there  is  not  a  perfect  relationship  between  indicators, 
events,  and  actual  enemy  intent.  The  enemy  will  be  expected 
to  use  deceptive  measures.  In  fact,  "An  integral  part  of 
templating  is  the  consideration  of  deception  events  associ¬ 
ated  with  each  course  of  action"  (TC30-27,  p.  4-11) .  As  a 
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result,  the  intelligence  team  will  be  forced  to  make  condi¬ 
tional  probability  judgments  about  what  indicators  and 
events  they  think  are  most  indicative  of  the  OPFOR  adoption 
of  different  courses  of  action.  Neither  TC30-27  nor  TC34-3, 
however,  provide  any  information  about  the  probabilistic 
relations  between  different  indications  and  different  OPFOR 
threats,  a  point  made  previously  when  discussing  TI  zone 
determination  and  evaluation. 

The  third,  and  final,  effort  in  threat  integra¬ 
tion  is  development  of  a  decision  support  template.  This 
template  "...  is  used  to  illustrate  enemy  probable  courses 
of  action  as  the  basis  for  comparing  friendly  courses  of  ac¬ 
tion"  (TC30-27,  p.  4-13).  Described  in  this  way,  the  deci¬ 
sion  support  template  "...  is  essentially  the  INTELLIGENCE 
ESTIMATE  in  graphic  format"  (TC30-27,  p.  1-6).  It  repre¬ 
sents  the  analysts'  most  up-to-date  estimate  about  the  rela¬ 
tive  likelihood  of  the  enemy's  potential  courses  of  action. 
Again,  however,  TC30-27  and  TC34-3  fail  to  indicate  how  the 
judgments  underlying  this  graphic  representation  are  to  be 
made,  or  in  fact  are  made,  by  individual  analysts.  Psycho¬ 
logical  research  suggests  that  this  may  result  in  decision 
support  templates  that  (1)  are  not  as  accurate  as  they  could 
be,  and  (2)  do  not  facilitate  communication  between  analysts 
and  commanders  as  much  as  they  could  otherwise. 

1.1.2  Template  revision  -  Up  to  this  point  there  has 
been  no  discussion  of  the  dynamic  nature  of  the  proposed  IPB 
templating  process.  The  templating  process  is  not  supposed 
to  stop  with  the  first  decision  support  template.  The  deci¬ 
sion  support  template  at  the  end  of  one  cycle  of  the  process 
represents  the  situation  template  at  the  beginning  of  the 
next  cycle.  The  event  templates  and  event  matrices  are  used 
to  convey,  as  quickly  as  possible,  revised  estimates  about 
the  likelihood  of  different  OPFOR  courses  of  action.  These 
revised  estimates  of  enemy  intent  are  represented  in  the 


decision  support  template.  "Used  properly,  they  [i.e.,  the 
templates)  provide  for  continuing  integrative  analysis  of 
OPFOR  capabilities,  vulnerabilities  and  courses  of  action" 
(TC30-27,  p.  1-5) .  Figure  1-1  provides  a  schematic  repre¬ 
sentation  of  this  iterative  process. 

The  dynamic  nature  of  the  proposed  templating 
process  is  heavily  dependent  on  a  complex  judgment  process. 
First,  intelligence  analysts  using  the  proposed  IPB  analysis 
process  must  identify  the  initial  hypotheses  regarding  pos¬ 
sible  enemy  courses  of  action,  and  if  possible,  make  an  initial 
estimate  of  the  most  probable  course  of  action.  As  discussed 
above,  this  requires  consideration  of  how  the  enemy  generally 
makes  judgments  of  intent,  in  addition  to  an  evaluation  of 
how  physical  terrain  and  weather  factors  favor  different 
courses  of  action.  Second,  intelligence  analysts  using  the 
proposed  IPB  process  must  indicate  the  events  that  are 
likely  to  be  observed  for  each  course  of  action.  The  infor¬ 
mation  within  all  the  event  templates  represents,  in  quali¬ 
tative  terms,  likelihood  ratios  that  indicate  how  much  more 
likely  certain  events  are  to  be  observed  for  one  course  of 
action  than  another.  Third,  the  analysts  must  use  the  many 
pieces  of  potentially  fallible  data  reported  in  the  event 
analysis  matrices  (and  other  sources)  to  revise  their  initial 
hypotheses  about  the  enemy's  most  likely  course  of  action. 

The  data  reported  in  the  event  analysis  matrices  also  repre¬ 
sent  conditional  probabilities,  for  certain  events  have  a 
higher  probability  of  being  observed  only  when  the  enemy  is 
actually  taking  a  particular  course  of  action.  The  events 
reported  in  the  event  analysis  matrices  are  matched  with 
those  hypothesized  in  the  event  templates  in  order  to  de¬ 
velop  a  revised  estimate  of  enemy  intent,  which  is  repre¬ 
sented  in  the  decision  support  template.  The  decision  sup¬ 
port  template  is  now  the  situation  template  for  the  next 
iteration. 
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SCHEMATIC  REPRESENTATION  OF  THE  DYNAMIC  NATURE 
OF  THE  PROPOSED  IPB  TEMPLATING  PROCESS 
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1. 2  Causal  Inference  Research  Findings  and  Computer-based 

Judgment  Aids  Applicable  to  IPB 

This  section  is  divided  into  two  parts.  The  first  part 
describes  causal  inference  research  findings  and  computerized 
judgment  aids  relevant  to  the  development  of  the  initial  de¬ 
cision  support  template.  The  second  part  describes  research 
and  aids  relevant  to  the  revision  of  decision  support  tem¬ 
plates  on  the  basis  of  newly  collected  data  represented  in 
the  event  template  and  event  analysis  matrix. 

1.2.1  Research  and  aids  applicable  to  the  development 
of  the  initial  decision  support  template  -  Research  on 
causal  inference  with  experts  in  different  fields  indicates 
that  the  level  of  judgmental  accuracy  and  interpersonal 
agreement  is  a  direct  function  of  the  characteristics  of  the 
judgmental  task  facing  the  experts.  This  conclusion  is  fur¬ 
ther  supported  by  controlled  laboratory  research,  which  has 
shown  that  (1)  judgmental  accuracy  and  agreement  can  be  in¬ 
creased  or  decreased  by  manipulating  characteristics  of  the 
task,  and  (2)  that  relatively  high  levels  of  accuracy  and 
agreement  can  be  maintained  under  conditions  that  normally 
prevent  them,  by  using  judgment  aids  that  provide  persons 
with  information  about  task  characteristics. 

Causal  inference  tasks  can  be  characterized  in 
terms  of  their  formal  characteristics.  Formal  task  charac¬ 
teristics  refer  to  the  statistical  relationships  among  task 
variables.  Research  on  the  following  two  task  characteris¬ 
tics  is  of  particular  relevance  to  effective  intelligence 
analysis:  (a)  the  predictability  of  individual  attributes, 

and  (b)  the  total  predictability  of  all  the  attributes  in 
the  inference  task.  Regarding  the  former,  researchers  have 
found  achievement  lowest  in  tasks  in  which  individual  attri¬ 
butes  have  equal  predictability,  which  is  frequently  the 
case  in  reviewing  Tactical  Order  of  Battle  according  to 
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Bowen  et  al.  (1975) .  Regarding  the  latter,  researchers  have 
found  a  positive  linear  function  between  achievement  and 
overall  task  predictability;  lower  task  predictability  re¬ 
sults  in  lower  achievement.  This  occurs  because  people  be¬ 
come  less  consistent  in  their  judgments  as  the  task  becomes 
less  predictable,  even  when  they  know  how  to  perform.  This 
finding  is  of  particular  importance  to  intelligence  analysts 
because  the  generally  low  level  of  overall  predictability  in 
some  intelligence  tasks  suggests  that  analysts  may  perform 
suboptimally  because  they  do  not  have  computerized  aids  for 
utilizing  their  knowledge  of  enemy  activities  in  a  rapid, 
highly  consistent  fashion. 

Controlled  laboratory  research  has  shown  that 
formal  task  characteristics  also  affect  interpersonal  agree¬ 
ment.  Two  task  characteristics  of  particular  relevance  to 
intelligence  analysis  are  overall  task  predictability  and 
attribute  intercorrelations.  Regarding  the  former,  the 
lower  the  level  of  task  predictability,  the  lower  the  level 
of  interpersonal  understanding  and  agreement.  This  occurs 
because  people  try  to  predict  the  randomness  (i.e.,  uncer¬ 
tainty)  in  the  causal  inference  task;  consequently,  they 
make  it  difficult  for  the  other  person  to  learn  how  they 
make  their  judgments,  and  thereby,  how  to  resolve  the  dis¬ 
agreement.  Attribute  intercorrelations  also  reduce  inter¬ 
personal  learning.  For  example,  if  two  attributes  (or 
factors)  have  a  high  positive  correlation,  people  can  rely 
on  different  attributes  and  still  have  high  levels  of  accu¬ 
racy  and  agreement.  If  the  situation  should  chanae,  how¬ 
ever,  such  that  the  attributes  are  no  longer  correlated,  the 
two  persons  will  disagree.  Furthermore,  the  dispute  will  be 
difficult  to  resolve  because  both  participants  will  point  to 
their  recent  successes,  although  only  one  of  the  individuals 
may  actually  know  the  valid  indicator. 


Laboratory  research  has  shown  that  computerized 
judgment  aids  that  quantitatively  describe  how  people  com¬ 
bine  information  on  multiple  attributes  into  an  overall 
judgment  greatly  enhances  judgmental  skill  and  interpersonal 
agreement.  Computer  graphics  devices  have  been  used  widely 
as  judgment  aids  for  presenting  cognitive  feedback,  feedback 
that  permits  persons  to  compare  the  formal  properties  of 
their  judgmental  system  with  those  of  the  task.  Cognitive 
feedback  provided  by  such  displays  has  been  shown  to  lead  to 
both  a  faster  rate  of  learning  and  a  higher  level  of  achieve¬ 
ment  (r  )  than  outcome  feedback  in  tasks  that  varied  the 
— a 

(a)  differences  in  the  individual  predictabilities  of  the 
attributes,  and  (b)  the  overall  level  of  task  predictability. 
While  this  finding  is  due  in  some  part  to  greater  knowledge 
acquisition  with  cognitive  rather  than  outcome  feedback,  it 
is  primarily  due  to  higher  cognitive  control  with  cognitive 
feedback.  This  finding  strongly  supports  our  position  that 
computerized  judgment  aids  will  improve  the  judgmental  accu¬ 
racy  of  experienced  intelligence  analysts  using  IPB  by 
greatly  increasing  their  ability  to  implement  their  knowl¬ 
edge.  Furthermore,  such  aids  will  improve  the  training  of 
analysts  learning  IPB. 

Computerized,  multi-attribute  judgment  aids  en¬ 
hance  interpersonal  learning  because  they  help  people  over¬ 
come  the  inaccuracy  and  inconsistency  of  verbal  self-reports. 
Verbal  reports  are  often  inaccurate  because  people  inaccu¬ 
rately  estimate  the  weight  they  place  on  various  attributes. 
Research  indicates  that  people  tend  to  underestimate  the 
weight  they  place  on  important  factors  and  overestimate  the 
weight  they  place  on  unimportant  factors  when  compared  to  a 
quantitative  analysis  of  their  judgments.  People  often 
apply  their  judgments  inconsistently  because  of  task  unpre¬ 
dictability  (or  uncertainty) .  Research  has  found  that  in¬ 
consistency  leads  to  interpersonal  misunderstanding  and  con¬ 
flict,  for  the  greater  the  inconsistency  in  one's  judgment 
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the  more  difficult  it  is  for  others  to  learn  how  one  makes 
one's  judgment.  Furthermore,  research  has  shown  that  judg¬ 
ment  aids  that  quantitatively  describe  (1)  how  different 
people  weight  the  different  attributes,  (2)  the  overall 
level  of  consistency  for  each  person's  judgments,  and 
(3)  the  implications  of  interpersonal  differences  in  judg¬ 
ment,  will  enhance  interpersonal  understanding  and  reduce 
interpersonal  conflict. 

Multi-attribute  utility  assessment  (MAUA)  tech¬ 
niques  and  judgment  aids  can  be  used  to  help  intelligence 
analysts  develop  the  initial  decision  support  template. 

They  provide  (1)  a  logically  defensible  conceptual  structure 
for  structuring  the  factors  used  in  estimating  enemy  intent, 
(2)  techniques  for  estimating  the  necessary  trade-offs  in¬ 
herent  in  estimating  enemy  intent,  (3)  an  analytical  pro¬ 
cedure  for  combining  the  multiple  trade-off  judgments  into 
an  overall  assessment  of  the  likelihood  of  different  enemy 
courses  of  actions,  (4)  a  means  of  systematically  investi¬ 
gating  the  implications  of  differences  of  opinion  in  judg¬ 
ment,  and  (5)  a  means  of  conveying  all  this  information 
pictorially,  thus  providing  an  important  adjunct  to  the 
proposed  pictorial  format  of  the  decision  support  template. 

A  computerized  judgment  aid  called  ENCOA  (Enemy 
Courses  of  Action)  has  been  developed  to  assist  Staff  Intel¬ 
ligence  Officers  (G2/S2s)  develop  the  initial  decision  sup¬ 
port  template;  see  Patterson  and  Phelps  (1980)  for  a  complete 
description.  ENCOA  provides  analysts  with  a  systematic  pro¬ 
cedure  for  evaluating  each  potential  OPFOR  course  of  action 
on  twenty-four  factors  affecting  enemy  intent.  The  factors 
incorporated  into  ENCOA  not  only  include  the  order  of  battle, 
terrain,  and  weather  factors  discussed  in  circulars  TC30-27 
and  TC34-3,  but  other  factors  that  any  tactical  commander 
would  consider  when  selecting  a  course  of  action.  These 
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factors  are  grouped  into  the  five  categories  indicated  in 
Figure  1-2. 


When  developing  the  initial  decision  support 
template,  intelligence  analysts  can  use  ENCOA  to  assess  each 
OPFOR  course  of  action  on  each  of  the  factors  comprising  the 
five  groups  of  factors.  Thus,  each  course  of  action  would 
receive  a  score  on  Terrain,  U.S.  Force  Factors,  OPFOR  Force 
Factors,  Weather  Factors,  and  Risk  Factors,  where  the  score 
represented  the  utility  of  the  course  of  action  to  an  OPFOR 
commander.  Then,  these  scores  can  be  differentially  weighted 
to  represent  the  trade-offs  of  the  OPFOR  commander.  The 
scores  and  weights  can  be  combined  to  predict  the  overall 
utility  of  each  course  of  action  to  the  OPFOR  commander.  In 
addition,  ENCOA  can  be  used  to  quantitatively  and  pictorially 
present  the  scores,  weights,  overall  utilities,  and  subse¬ 
quent  sensitivity  analyses  to  the  friendly  commander.  Such 
information  describing  (1)  hov:  the  analysts  reached  their 
conclusions  about  the  most  likely  OPFOR  courses  of  action, 
as  well  as  (2)  the  implications  of  differences  in  opinion 
between  the  analysts,  represent  an  important  adjunct  to  the 
decision  support  template. 


1.2.2  Research  and  aids  applicable  to  the  revision  of 
decision  support  templates  -  The  iterative,  templating  pro¬ 
cess  proposed  in  circulars  TC30-27  and  TC34-3  can  be  repre¬ 
sented  quantitatively  by  Bayes'  Theorem,  which  is  shovm  in 
equation  [ 1] . 
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[.  Terrain  factor; 

Aj  related  to  alsslon  accomplishment 
and  conalderlng  current  OPFOR  doctrine,  acore 
each  OPFOR  courae  of  action  In  terms  of  how 


well 

it: 

1.1 

Exploits  field  of  fire  efforded  by  ter- 
train  features. 

1.2 

Exploits  cover  and  concealment  afforded 
by  terrain  features. 

1.3 

Exploits  mobility  provisions  due  to  cer¬ 
tain  features. 

1.4 

Accomplishes  rapid  seizure  or  denial  of 
key  cerrain. 

1.5 

Exploit*  observation  provisions  of  terrain. 

1.6 

Exploits  or  accommodates  natural  and  artificial 
obstacles. 

II. 

I'.S. 

Force  Factors 

As  related  to  mission  accomplishment  and  con¬ 
sidering  current  U.S.  doctrine,  acore  each  OPFOR 
course  of  action  in  terms  of  how  well  it  exploits 
what  you  know  or  estimate  about : 

2.1 

U.S.  disposition. 

2.2 

U.S.  strength  and  condition. 

2.3 

U.S.  reserves. 

2.4 

U.S.  logistic  support. 

2.5 

Probable  U.S.  actions ''reactions  . 

2.6 

U.S.  command  and  control  capabilities/ 
vulnerabilities. 

III .  Opposing  Force  Fagtora 

Aa  related  to  mission  accomplishment  and  con¬ 
alderlng  current  OPFOR  doctrine,  acore  each  OPFOR 
courae  of  action  In  terms  of  how  well  It  exploit* 
or  accommodates : 

3.1  OPFOR  current  disposition. 

3.2  OPFOR  strength  and  condition. 

3.3  OPFOR  reserves. 

3.4  OPFOR  logistic  support. 

3.5  OPFOR  command  and  control  capabilities/ 
vulnerabilities. 

IV .  Weather  Factors 

As  related  to  mission  accomplishment,  score 
each  OPFOR  course  of  action  In  terms  of  how  veil 
It  exploits: 

4.1  Observation/visibility  conditions  forecast 
to  exist  due  to  weather. 

4.2  Cover  and  concealment  conditions  forecast 
to  exist  due  to  weather. 

4.3  Mobility  conditions  forecast  to  exist  due 
to  weather . 

4.4  Effect  of  extreme  conditions  ef  forecast 
weather  on  personnel  and  equipment 
effectiveness . 

V .  Rlsi  Factors 

As  related  to  mission  accomplishment ,  score 
each  OPFOR  course  of  action  in  terms  of: 

5.1  Ability  to  cope  with  surprises  in  terms 

of  I'.S.  strength  or  U.S.  ac t lons/teact ions . 

5.2  Freedom  from  dependence  on  forces  not 
under  own  control. 


5.3  Freedom  from,  critical  dependence  on 
surprise  or  deception. 

5.4  Suitability  under  unexpected  adverse 
weather  conditions. 


Figure  1-2 
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The  situation  template  is  represented  by  the  prior  probabili¬ 
ties,  which  indicate  the  relative  likelihood  of  the  different 
OPFOR  courses  of  action  (i.e.,  hypotheses)  under  considera¬ 
tion.  The  event  templates  and  event  matrices  are  represented 
by  the  conditional  probabilities,  which  indicate  the  relative 
likelihood  that  certain  events  support  particular  courses  of 
action.  The  posterior  probabilities  are  represented  by  the 
decision  support  template,  which  indicates  the  revised  like¬ 
lihood  of  the  courses  of  action  (i.e. ,  hypotheses)  on  the 
basis  of  observed  data.  This  new  estimate  of  enemy  intent 
is  then  input  to  friendly  tactical  decision  making  and 
subsequent  action. 

Reviews  of  psychological  research  in  which  sub¬ 
jects'  final  probability  estimates  have  been  compared  with 
those  prescribed  by  Bayes'  Theorem  have  shown  consistently 
that  humans  are  suboptimal  processors  of  probabilistic  in¬ 
formation.  Although  they  typically  revise  their  opinions  in 
the  same  direction  as  Bayes'  Theorem,  they  do  not  revise 
them  enough;  they  are  conservative.  This  finding  could  have 
great  implications  for  IPB,  for  if  analysts  using  IPB  are 
conservative  information  processors,  then  they  are  not  draw¬ 
ing  implications  from  the  data  as  fast  as  they  could  be  with 
Bayes1  Theorem.  Their  estimates  about  enemy  courses  of  ac¬ 
tion  may  well  be  suboptimal  because  they  will  not  have  suf¬ 
ficiently  revised  their  opinions  to  take  full  account  of  the 
certainty  in  the  data.  Consequently,  the  entire  templating 
process  will  not  convey  as  much  information  to  commanders  as 
it  should.  The  time  available  for  friendly  tactical  deci¬ 
sion  planning  and  implementation  may  be  reduced  considerably 
if  intelligence  analysts  are  conservative  information  pro¬ 
cessors  . 

The  potential  implications  of  Bayesian  research 
for  IPB  are  compounded  by  the  fact  that  intelligence  analysts 
rely  on  language  instead  of  numerical  estimates  to  convey 
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uncertain  information.  It  is  impossible  to  directly  trans¬ 
late  qualitative  expressions  of  uncertainty  such  as  "very 
likely"  into  probability  values.  For  while  most  people 
would  agree  that  "very  likely"  means  a  probability  greater 
than  0.5,  there  is  no  general  agreement  of  how  much  more 
than  0.5  the  probability  is  (0.8?,  0.9?,  0.99?).  The  lack 
of  agreement  in  the  use  of  language  to  convey  probability 
estimates  has  been  demonstrated  with  intelligence  analysts 
in  a  number  of  experiments. 

Research  is  beginning  to  identify  why  individuals 
are  conservative  information  processors.  One  reason,  for 
example,  is  that  people  tend  to  search  for  confirming  rather 
than  disconf irming  evidence  of  alternative  hypotheses.  This 
is  often  a  suboptimal  data  collection  and  revision  strategy 
because  data  often  can  confirm  many  hypotheses  at  the  same 
time;  disconf irming  evidence,  on  the  other  hand,  can  quickly 
eliminate  a  hypothesis  from  consideration.  Another  reason 
why  individuals  are  conservative  information  processors  is 
that  they  tend  to  use  simple  rules  or  judgment  heuristics 
instead  of  the  axioms  of  probability  theory  when  making  sub¬ 
jective  probability  estimates.  As  a  result  of  learning  why 
individuals  are  conservative  information  processors,  re¬ 
searchers  have  been  able  to  develop  training  methods  and 
judgment  aids  to  help  people  make  more  accurate  probability 
estimates . 


General  judgment  aids,  called  Probabilistic  In¬ 
formation  Processing  (PIP)  systems  have  been  developed  to 
ensure  that  human  judgment  is  consistent  with  Bayes*  Theorem 
for  a  number  of  intelligence  problems  (e.g.,  see  Peterson 
et  al.,  1976).  Such  a  judgment  aid  is  now  being  developed 
to  help  tactical  intelligence  analysts  revise  their  judgment 
about  the  most  likely  OPFOR  course  of  action  on  the  basis  of 
new  information.  The  aid  will  probably  operate  in  the  fol¬ 
lowing  manner; 
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(1)  the  analysts  define  the  n  different  enemy  courses 
of  action  (COAs)  under  consideration  (i.e.,  enter 
a  brief  [<  10  character  title]); 

(2)  the  analysts  enter  a  set  of  prior  probabilities 
(or  odds)  for  the  n  potential  courses  of  action 
(this  could  be  a  set  of  n  probabilities  summing  to 
1.0  or  a  set  of  n-1  likelihood  ratios  assessed 
relative  to  a  selected  COA)  ; 

(3)  for  a  given  datum,  the  analysts  input  a  brief 
title  and  the  probability  (or  odds  in  terms  of  a 
likelihood  ratio)  of  that  datum  conditional  upon 
each  COA  being  considered; 

(4)  the  analysts  inspect  the  posterior  probabilities 
(or  odds) ; 

(5)  they  revise  any  posterior  probabilities  that  are 
counter-intuitive;  and 

(6)  the  analysts  report  on  potential  enemy  COAs  based 
on  the  probabilities  after  step  5  or  return  to 
step  3  if  there  are  additional  data. 

The  ability  to  use  a  Bayesian  framework  to  rep¬ 
resent  the  different  judgments  inherent  in  revising  the  IPB 
templates  is  illustrated  in  the  above  steps.  Step  1  repre¬ 
sents  the  different  OPFOR  courses  of  action  represented  in 
the  situation  template.  Step  2  represents  the  relative 
likelihood  of  these  actions  at  the  end  of  one  iteration  of 
the  IPB  analysis  process.  Step  3  represents  the  judgments 
in  the  event  templates  and  event  matrices,  which  indicate 
the  relative  likelihood  of  having  collected  the  newly  ac¬ 
quired  data  on  the  basis  of  the  enemy  actually  taking  dif¬ 
ferent  courses  of  action.  Steps  4,  5,  and  6  represent  the 
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intelligence  analysts*  estimates  of  enemy  intent,  as  repre¬ 
sented  graphically  in  the  decision  support  template. 

The  aid  is  being  designed  so  that  it  can  be 
readily  integrated  into  the  proposed  iterative  IPB  analysis 
process.  Consequently,  it  will  provide  an  important  adjunct 
to  the  development  and  utilization  of  IPB  templates.  The 
decision  aid  is  scheduled  for  initial  evaluation  during 
1981. 
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2.0  INTELLIGENCE  PREPARATION  OF  THE 
BATTLEFIELD:  DEPENDENCE  ON  JUDGMENT 


"Everyone  complains  of  his  memory,  no  one  of  his 

judgment."  La  Rochefoucauld. 

Techniques  for  gathering  intelligence  data  have  been 
steadily  increasing  in  sophistication,  particularly  in  the 
areas  of  electromagnetic  and  imagery  intelligence.  In  con¬ 
trast,  techniques  for  interpreting  the  implications  of  in¬ 
telligence  data  in  terms  of  enemy  intent  appear  to  have  im¬ 
proved  at  a  more  modest  rate  (e.g.,  Bowen  et  al.,  1975). 
Intelligence  Preparation  of  the  Battlefield,  as  described  in 
Circulars  TC30-27  (1978)  and  TC34-3  (in  press) ,  represents  a 
major  step  toward  improving  procedures  for  correlating  data 
with  enemy  intent  and/or  capabilities. 

Intelligence  Preparation  of  the  Battlefield  (IPB)  is  an 
analytical  tool  to  "...help  the  analyst  'visualize'  the 
variables  of  how  the  enemy  might  fight  on  a  specific  piece 
of  terrain  at  certain  times"  (TC30-27,  1-5).  This  is  accom¬ 
plished  through  " templating , "  using  a  series  of  graphic  il¬ 
lustrations  of  an  enemy  capability  drawn  as  an  overlay  over 
a  map.  The  process  whereby  input  (new  intelligence  data)  is 
translated  into  output  (templates)  necessarily  depends  on 
analysts'  judgments  about  the  implications  of  multiple 
pieces  of  potentially  fallible  data.  TC30-27  and  TC34-3, 
however,  provide  little  information  about  how  intelligence 
analysts  should  make  these  judgments.  This  omission  may 
limit  the  usability  and  accuracy  of  IPB  as  it  is  proposed  in 
these  circulars.  The  purpose  of  this  paper  is  to:  (1)  dem¬ 
onstrate  how  the  execution  of  IPB  procedures  depends  on 
human  judgment,  and  (2)  indicate  how  the  incorporation  of 
decision-analytic  procedures  and  computerized  judgment  aids 
can  improve  the  quality  of  judgment  in  IPB. 
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Over  the  last  twenty-five  years,  thousands  of  scientific 
studies  of  human  judgment  and  decision  making  have  reached 
one  basic  conclusion:  human  judgment  is  limited.  The  con¬ 
ceptualization  of  human  rationality  as  limited  or  "bounded,” 
of  even  flawed,  is  a  relatively  new  intellectual  idea,  one 
initially  developed  in  large  part  by  Herbert  Simon,  the  1978 
Nobel  laureate.  As  the  quote  by  La  Rochefoucauld  suggests, 
one's  cognitive  ability  to  decide  rationally  is  seldom  ques¬ 
tioned.  In  fact,  as  TC30-27  illustrates,  the  question  may 
never  be  raised.  People  may  complain  that  their  decisions 
were  poor  because  they  didn't  have  enough  information  or  the 
right  information  or  management  information  system  to  keep 
track  of  all  the  information.  But  they  will  seldom,  if 
ever,  consider  that  they  just  didn't  have  the  cognitive  or 
logical  tools  to  make  a  good  decision.  Yet  scientific  re¬ 
search  suggests  that  this  is  often  the  case. 

Our  self-image  to  the  contrary,  judgments  of  experts 
and  lay  people  alike  have  been  shown  to  be  inaccurate  and 
subject  to  systematic  biases  under  laboratory  and  field  con¬ 
ditions.  We  quote  Einhorn  and  Hogarth  (1978)  on  this  point: 

"Although  the  study  and  cataloguing  of  judgmental 
fallibility  has  had  a  long  history  in  psychology 
(see,  e.g.,  Guilford,  1954,  chapter  12;  Johnson, 

1972) ,  an  accumulating  body  of  recent  research 
on  clinical  judgment,  decision-making,  and  proba¬ 
bility  estimation  has  documented  a  substantial 
lack  of  ability  across  both  individuals  and  situa¬ 
tions"  (Slovic,  Fischhoff,  &  Lichtenstein,  1977; 

Slovic  &  Lichtenstein,  1971)  p.  395. 

The  judgment  fallibility  referred  to  above  is  not  attribut¬ 
able  to  motivational  effects  such  as  wishful  thinking  or 
distortions  by  expected  payoffs.  Severe  errors  of  judgment 
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occur  even  when  participants  are  encouraged  to  be  accurate 
and  are  rewarded  for  the  correct  answer. 

In  light  of  the  scientific  findings,  judgment/decision 
researchers  have  begun  developing  decision-analytic  tech¬ 
niques  and  computerized  aids  to  help  decision  makers  improve 
their  cognitive  tools  and  skills.  (Kibler,  Watson,  Kelly, 
and  Phelps  [1978],  and  Patterson  and  Phelps  [1980]  have 
developed  such  procedures  to  aid  Army  G-2  and  G-3  staff.) 
These  techniques  and  aids  often  rely  on  the  principle  of 
divide  and  conquer.  That  is,  the  total  problem  is  divided 
into  a  series  of  structurally  related  parts.  The  decision 
maker  is  asked  to  evaluate  each  alternative  under  consider¬ 
ation  for  each  of  the  independent,  simpler  components  that 
comprise  the  larger  decision  problem.  Then,  computerized 
decision  aids  combine  all  the  judgments  in  order  to  provide 
an  overall  evaluation  of  each  alternative.  Many  of  the  aids 
also  provide  sensitivity  analysis,  thereby  permitting  deci¬ 
sion  makers  to  observe  the  effects  of  changing  their  judg¬ 
ments  on  each  alternative.  Thus,  decision-analytic  tech¬ 
niques  and  computerized  aids  "conquer"  the  global  decision 
problem  by  providing  an  analytical  means  of  expanding  the 
cognitive  limitations  of  unaided  human  judgment. 

The  analysis  process  proposed  for  IPB  can  be  improved 
by  incorporating  decision-analytic  techniques  and  computer¬ 
ized  decision  aids.  This  can  be  accomplished  because  the 
process  whereby  intelligence  data  is  translated  into  IPB 
templates  necessarily  depends  on  analysts'  judgments  about 
the  implication  of  the  information.  Scientific  research 
results  strongly  suggest  that  decision-analytic  techniques 
and  computerized  decision  aids  would  improve  both  (1)  the 
judgmental  accuracy  of  intelligence  analysts  developing  IPB 
templates,  and  (2)  the  communication  process  for  conveying 
the  reasons  for  these  judgments  to  tactical  commanders.  By 
improving  the  judgmental  process  inherent  within  IPB,  and 


the  means  for  conveying  the  reasons  for  these  judgments  to 
tactical  commanders,  the  broader  IPB  analysis  process  should 
be  improved  in  turn. 

2 . 1  Conceptual  Overview 

Figure  2-1  is  a  simplified  representation  of  the  judg¬ 
mental  nature  of  the  intelligence  process.  The  goal  of  in¬ 
telligence  analysis  is  to  predict  enemy  intent  or  capability 
(right  circle  in  Figure  2-1)  .  Because  the  enemy  intent  is 
unknown  (left  circle.  Figure  2-1),  intelligence  analysts 
mast  base  their  predictions  on  information  and  indicators 
that  serve  as  cues  to  determining  the  unknown  enemy  intent. 
Intelligence  predictions  are  accurate  to  the  extent  they 
match  the  unknown  enemy  intent  as  shown  in  Figure  2-1.  Ac¬ 
curacy  is  thus  a  function  of:  (1)  how  reliably  different 
cues  reflect  enemy  intent  and  (2)  the  degree  to  which  ana¬ 
lysts  base  their  predictions  on  the  most  reliable  cues.  The 
identification  and  integration  of  these  cues  to  form  the  in¬ 
telligence  estimate  depends  totally  on  the  subjective  judg¬ 
ment  of  the  intelligence  analyst. 

The  subjective  nature  of  intelligence  analysis  becomes 
even  more  obvious  when  more  realistic  representation  of  the 
intelligence  process,  such  as  those  represented  in  Figures 
2-2  and  2-3,  are  considered.  In  particular,  there  are  three 
factors  complicating  the  analyst’s  judgment  shown  in  Figure 
2-2.  First,  the  relationship  between  the  information  and 
indicators  is  probabilistic;  that  is,  indicators  or  informa¬ 
tion  correspond  to  enemy  intent  less  than  100%  of  the  time, 
causing  the  indicators  to  vary  in  reliability.  Second,  in¬ 
telligence  analysts  may  not  use  some  indicators  regardless 
of  the  reliability.  Third,  some  information,  not  related  to 
the  enemy  intent,  may  influence  the  analyst's  prediction. 
Analysts  must  be  able  to  perceive  the  relative  reliability 
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of  indicators  and  be  able  to  use  only  the  most  reliable  ones 
in  their  analysis,  ignoring  the  unreliable  indicators. 

At  a  more  conceptual  level,  the  task  facing  intelligence 
analysts  is  one  of  causal  inference.  In  fact.  Figure  2-1 
merely  represents  a  minor  modification  of  the  "lens  model," 
originally  developed  by  Brunswik  (1952,  1956),  to  describe 
pictorially  the  judgmental  process  of  causal  inference.  In 
considering  the  characteristics  of  causal  environments, 
Brunswik  discussed  two  concepts  of  particular  importance 
when  considering  the  judgmental  nature  of  intelligence 
analysis.  First,  the  environment  is  a  causal  texture  with 
different  events  being  dependent  on  each  other.  As  Figure 
2-1  implies,  cause  and  effect  is  conceived  of  as  an  "if- 
then"  contingency;  X.  is  clearly  dependent  on  Y  ,  assuming, 
of  course,  that  Y^  occurs  first  in  time.  Consequently,  a 
given  can  be  considered  to  be  a  local  representative  of 
(or  cue  to)  the  more  distal  event  Y  .  Second,  the  depen- 
dency  between  a  given  cue  X.  and  a  given  distal  state  Y  ,  is 

”1  “O 

almost  always  to  some  extent  equivocal  or  probabilistic? 

Hard  and  fast  one-to-one  relations  between  a  distal  state 
and  cue  do  not  always  exist.  As  a  result,  a  cue  can  be 
causally  connected  with  more  than  one  proximal  stimulus. 

The  environment  therefore  ordinarily  involves  only  probable 
causes  and  probable  effects. 

This  situation  is  portrayed  by  Figure  2-3.  The  letters 
Yel  and  Ye2  indicate  that  there  are  two  possible  causes  of 
the  data,  or  effects,  observed  by  the  analyst.  The  dashed 
lines  indicate  the  probabilistic  nature  of  some  cause-effect 
relations.  For  example,  X2  is  only  related  to  Ye^;  there¬ 
fore,  it  is  an  unequivocal  predictor  of  The  question 

facing  analysts  and  commanders  is,  what  data  best  predict 
Y@^  and  what  data  best  predict  Yg2? 


The  proposed  IPB  analysis  process  essentially  intends 
to  answer  this  question  by  templating,  that  is,  by  using  a 
series  of  graphic  illustrations  of  an  enemy  capability  drawn 
as  an  overlay  for  a  map.  While  important,  this  advance  does 
not  go  far  enough,  for  it  fails  to  make  explicit  how  input, 
new  intelligence  data  (X^) ,  is  to  be  converted  into  output, 
templates.  This  translation  process  necessarily  depends  on 
judgment.  TC30-27  and  TC34-3  provide  little  information 
about  how  intelligence  analysts  should  make  these  judgments. 

As  noted  in  the  introduction,  we  believe  that  the 
proposed  IPB  analysis  process  can  be  strengthened  and  ex¬ 
panded  by  incorporating  decision-analytic  techniques  and 
computerized  aids.  The  next  section  of  this  report  is  di¬ 
rected  toward  identifying  the  different  types  of  judgment 
inherent  in  each  of  the  five  steps  in  the  proposed  analysis 
process.  Figures  2-1,  2-2,  and  2-3  provide  a  general  con¬ 
ceptual  reference  for  considering  judgments  of  enemy  intent. 

2 . 2  The  Role  of  Judgment  in  Each  of  the  Five  IPB  Steps 

"IPB  is  a  continuous  process  of  analysis  and  evaluation 
which  is  the  basis  of  intelligence  operations  planning... 

The  purpose  of  this  analysis  is  to  determine  and  evaluate 
enemy  capabilities,  vulnerabilities,  and  courses  of  action 
as  the  basis  for  friendly  operations  planning"  (TC30-27, 
p.  iii) .  To  accomplish  this  purpose,  the  following  five 
steps  in  IPB  analysis  are  proposed  in  TC30-27:  (1)  mission 

and  threat  evaluation,  (2)  TI  zones  determination  and  evalu¬ 
ation,  (3)  terrain  analysis,  (4)  weather  analysis,  and  (5) 
threat  integration.  The  next  section  of  this  report  is 
directed  toward  identifying  the  different  types  of  judgment 
inherent  in  each  of  these  five  steps,  in  turn. 

2.2.1  Mission  and  threat  evaluation  -  This  is  the 
first  step  of  the  proposed  IPB  analysis  process.  Mission 
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evaluation  establishes  the  area  of  operation  and  identifies 
the  general  battlefield  scenario.  Threat  evaluation  of  doc¬ 
trine,  tactics,  capabilities,  and  equipment  includes  a  re¬ 
view  of  the  following  order  of  battle  intelligence  holdings: 
composition,  disposition,  strength,  tactics,  training,  lo¬ 
gistics,  and  combat  effectiveness.  The  purpose  of  this  re¬ 
view  is  to  identify  the  gaps  in  collected  intelligence  data 
and,  thereby,  guide  future  collection  efforts. 

Opposing  force  (OPFOR)  doctrine  is  to  serve  as 
the  basis  for  knowing  and  understanding  the  potential  adver¬ 
sary  and,  therefore,  implementing  mission  and  threat  evalua¬ 
tion.  Intelligence  analysts  are  to  develop  "doctrinal  tem¬ 
plates"  that  indicate  how  the  enemy  likes  to  fight.  Figure  1 
in  TC34-3,  for  example,  shows  a  doctrinal  template  for  a  mo¬ 
torized  rifle  regiment  in  movement  to  contact.  Intelligence 
analysts  and  tactical  commanders  "seeing"  an  OPFOR  motorized 
rifle  regiment  in  this  disposition,  therefore,  should  con¬ 
clude  that  the  enemy's  intent  is  to  attack. 

The  doctrinal  template  thus  represents  a  standard 
against  which  order  of  battle  holdings,  such  as  disposition, 
can  be  reviewed.  If  the  intelligence  team  does  not  know, 
for  the  above  example,  the  disposition,  composition,  or  com¬ 
bat  effectiveness  of  the  OPFOR  motorized  rifle  regiment  in 
the  area  of  operation,  then  intelligence  data  must  be  col¬ 
lected  on  the  missing  order  of  battle  holdings.  Once  the 
data  are  collected,  the  team  can  convey  their  evaluation  of 
OPFOR  threat  to  the  commander  in  the  form  of  a  template  or 
otherwise.  What  Circulars  TC30-27  and  TC34-3  fail  to  point 
out,  however,  is  that  the  review  of  order  of  battle  holdings 
is  a  complex  judgmental  process.  The  intelligence  team  must 
evaluate  a  large  amount  of  incoming  data,  much  of  which  may 
be  unreliable,  in  order  to  decide  whether  it  "knows"  the 
order  of  battle  holdings.  In  addition,  they  must  integrate 


the  order  of  battle  holdings,  complete  or  not,  into  a  judg¬ 
ment  of  enemy  intent. 

The  judgmental  nature  of  order  of  battle  holdings 
is  noted  clearly  by  Bowen,  Halpin,  Russell,  and  Staniforth 
(1975)  in  Tactical  Order  of  Battle:  A  State  of  the  Art  Sur¬ 
vey.  Judgment  is  required  because  there  is  simply  no  alter¬ 
native  to  it.  We  quote  Bowen,  et  al.  (1975,  p.  45)  on  this 
point: 

"Each  OB  Factor  is  defined  in  terms  of  a  number  of 
information  elements  which  indicate  the  kind  of  data 
required  to  describe  the  status  of  that  factor  in 
narrative  terms.  No  specific  rules  exist,  either 
formal  or  heuristic,  for  the  evaluation  of  factors  or 
elements,  or  their  combination." 

In  fact,  the  lack  of  "specific  rules,"  or  in  terms  of  IPB, 
"templates,"  for  making  Order  of  Battle  judgments  is  repre¬ 
sented  in  each  of  the  following  "key  problems"  identified  by 
Bowen  et  al.  (1975,  pp.  59-60): 

"1.  There  are  no  standardized  methodologies  for  esti¬ 
mating  OB  factors.... 

2.  There  are  no  standardized  methodologies  for  in¬ 
corporating  OB  factors  into  the  products  of  OB 
intelligence . 

3.  There  is  no  methodology  for  estimating  and  report¬ 
ing  reliabilities  of  OB  factor  estimates  nor  the 
significance  of  levels  of  OB  factors  and  their 
elements  relative  to  general  descriptors  of  the 
state  of  enemy  forces,  such  as  Combat  Effective¬ 
ness. 
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There  is  no  realistic  and  generally  accepted  defi¬ 
nition  of  the  OB  factor  of  Combat  Effectiveness. 

A  serious  ambiguity  of  rationale  about  estimating 
enemy  Combat  Effectiveness  derives  from  the  point 
of  view  of  the  estimator.  (A  G2  thinks  in  terms 
of  the  enemy  force,  while  the  commander  of  G3  is 
concerned  with  the  enemy's  net  potential  effective¬ 
ness  relative  to  friendly  forces  in  the  existing 
circumstances. ) 

5.  There  is  no  indication  in  doctrine  of  the  relative 
importance  of  the  OB  factors,  or  of  their  inter¬ 
relationships  . 

6.  There  are  no  consistent,  validated  indicators, 
data  aggregates,  or  data  elements  for  the  various 
OB  factors  and  their  elements. 

7.  There  is  no  methodology  for  relating  the  elements 

of  an  OB  factor  to  each  other,  or  to  the  factor 

itself .  " 

Nor  does  TC30-27  or  TC34-3  offer  a  "methodology" 
for  making  the  judgment  inherent  in  threat  evaluation.  In¬ 
telligence  analysts  are  still  on  their  own  to  combine  data 
on  order  of  battle  holdings  as  they  see  fit.  And,  as  point 
#4  above  notes,  different  analysts  might  combine  the  data 
differently.  While  the  proposed  IPB  analysis  may  use  tem¬ 
plates  to  represent  the  output  of  this  judgmental  process, 
it  provides  no  means  for  representing,  pictorially  or  other¬ 
wise,  the  basis  for  the  judgments  underlying  threat  evalua¬ 
tion.  In  terms  of  Figure  2-2,  for  example,  TC30-27  and 

TC34-3  fail  to  provide  an  explicit  representation  of  the 
probable  relations  between  actual  enemy  intent  and  the  order 
of  battle  holdings  that  represent  indicators  of  intent  (X^ ; 
consequently,  it  offers  no  explicit  rules  or  heuristics 
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describing  how  order  of  battle  holding  (X^)  are  to  be  com¬ 
bined  into  judgments  of  enemy  intent.  Scientific  research 
on  causal  inference  indicates  that  the  accuracy  of  causal 
inferences  can  be  improved  by  providing  information  about 
probable  cause-effect  relations  represented  on  the  left  side 
of  the  model  in  Figure  2-2. 

2.2.2  Tactical  intelligence  zone  determination  and 
evaluation  -  This  is  the  second  step  in  the  proposed  IPB 
analysis  process.  Circular  TC30-27  proposes  three  TI  zones 
to  help  the  commander  at  the  company,  battalion/brigade,  and 
corps/division  levels,  respectively,  ”...  SEE  the  battle¬ 
field"  (TC30-27,  p.  27).  For,  according  to  TC30-27,  it  is 
the  intelligence  analyst's  job  to  make  sure  that  commanders 
know  what  the  intelligence  collection  and  processing  system 
can  and  cannot  do  prior  to  combat.  Toward  that  end,  TI  zone 
determination  and  evaluation  is  proposed  as  a  means  of  tell¬ 
ing  commanders  where  to  look  for  targets  and  indicators  of 
activities  that  confirm  the  enemy's  adoption  of  a  course  of 
action;  what  to  look  for,  that  is,  the  critical  indicators 
that  must  be  seen  by  a  certain  time;  when  in  the  battle  se¬ 
quence  to  look  for  them;  and  what  data  collection  tools  to 
look  with.  Circular  TC34-3  also  emphasizes  these  points, 
but  refers  to  the  TI  zones  as  "areas  of  interest/influence" 
(see  Chapter  3) . 

Step  2,  therefore,  is  supposed  to  identify  how 
the  OPFOR  threat  evaluated  in  Step  1  should  look  in  the  bat¬ 
tlefield,  in  general.  Circulars  TC30-27  and  TC34-3  accom¬ 
plish  this,  however,  only  to  a  limited  degree.  There  is, 
for  example,  no  discussion  of  the  general  indicators  for 
different  OPFOR  threats.  Yet  research  by  Johnson,  Spooner, 
and  Jaarsma  (1977)  suggests  that  this  would  be  a  valuable 
addition  for  tactical  commanders.  They  found  that  a  sample 
of  forty-three  captains  in  the  Intelligence  Officers  Advanced 


Course  knew  only  nineteen  of  the  forty-nine  separate  indi¬ 
cations  listed  in  Field  Manual  30-5,  Combat  Intelligence, 
for  the  four  separate  courses  of  action  to  be  evaluated  in 
Step  1  -  attack,  defense,  delay,  and  withdrawal. 

Just  as  in  the  case  of  threat  evaluation,  the 
judgmental  conceptualizations  shown  in  Figures  2-1,  2-2, 
and  2-3  represent  the  basic  judgment  problem  facing  analysts 
during  TI  zone  evaluation.  Doctrinal  templates  could  be  de¬ 
veloped  to  help  commanders  visualize  the  general  indicators 
representative  of  different  OPFOR  threats  at  different  TI 
zones.  In  addition,  decision  aids  could  be  developed  to 
assist  in  TI  zone  evaluation.  Since  individual  indications 
are  not  perfect  predictors  of  enemy  intent,  analysts  and 
commanders  must  know  the  relative  accuracy  of  individual  in¬ 
dicators  for  each  of  the  different  OPFOR  courses  of  action. 
Furthermore,  they  must  know  how  to  combine  individual  indi¬ 
cators  into  a  global  assessment  of  threat.  TC30-27  provides 
no  indication  of  how  this  is  to  be  accomplished  in  TI  zone 
evaluation.  Causal  inference  research  indicates  that  ex¬ 
plicit  information  about  the  left-hand  side  of  the  causal 
inference  represented  in  Figures  2-1,  2-2,  and  2-3,  greatly 
improves  the  judmental  accuracy  of  individuals. 

2.2.3  Terrain  and  weather  analysis  -  It  is  interesting 
to  note  that  Circulars  TC30-27  and  TC34-3  do  tell  analysts 
how  to  make  judgments  about  the  military  aspects  of  terrain 
and  weather,  the  third  and  fourth  steps,  respectively  in  the 
proposed  IPB  analysis  process.  The  determination  of  avenues 
of  approach  into  the  commander's  TI  zone  is  basic  to  terrain 
analysis.  To  accomplish  this,  the  IPB  analysis  team  must 
focus  its  analytical  efforts  on  the  following  five  military 
aspects  of  terrain:  observation  in  fields  of  fire,  conceal¬ 
ment  and  cover,  obstacles,  key  terrain,  and  avenues  of  ap¬ 
proach.  Toward  this  end,  the  circular  contains  a  factor  and 
subfactor  analysis  matrix,  and  a  list  of  terrain  requirements. 
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This  information  defines  the  relationships  between  the 
physical  aspects  of  terrain,  such  as  surface  configuration, 
vegetation,  rock  types  and  soils,  and  specific  military  as¬ 
pects  of  terrain,  such  as  observation  and  fields  of  fire, 
cover,  concealment,  and  obstacles.  Thus,  it  shows  the  ana¬ 
lysts  how  to  make  judgments  about  the  military  aspects  of 
terrain.  Such  analytical  efforts  facilitate  subsequent  "mo¬ 
bility  analysis"  and  "line-of-sight  analysis,"  the  result  of 
which  is  a  series  of  map  overlays  that  identify  the  possible 
OPFOR  avenues  of  approach  and  the  ..."  one  avenue  of  approach 
(mobility  corridor)  more  favorable  than  the  others"  (TC30-27, 
p.  3-26) .  They  also  provide  a  retraceable  procedure  for 
telling  the  commander  how  these  judgments  were  arrived  at  by 
the  analysts. 


As  in  the  case  of  terrain  analysis,  the  IPB 
analysis  process  addresses  the  effects  of  weather  on  ground 
and  air  mobility  and  line-of-sight.  In  a  similar  fashion, 
Circulars  TC30-27  and  TC34-3  address  in  considerable  detail 
how  the  "weather  factor  analysis  matrix"  and  "weather 
parameter-user  matrix"  are  to  be  used  to  relate  weather 
factors  to  a  wide  range  of  military  operations.  The  result 
of  Step  4  is  a  set  of  overlays  that  integrate  terrain  and 
weather  data  in  a  manner  that  conveys  their  possible  inter¬ 
active  effects  on  the  capability  of  forces  to  move,  shoot 
and  communicate  within  the  different  mobility  corridors  in 
the  situation  under  consideration.  The  word  "possible"  is 
underlined  to  emphasize  the  uncertain  nature  of  weather;  the 
best  mobility  corridor  if  it's  dry  can  be  the  worst  one  if 
it  rains. 

2.2.4  Threat  integration  -  This  is  the  fifth  and  final 
step  in  the  proposed  IPB  analysis  process.  "The  objective 
of  threat  integration  is  to  relate  "how  the  OPFOR  likes  to 
fight"  to  a  specific  terrain  and  weather  scenario  as  the 
basis  for  determining  "how  the  enemy  might  have  to  fight" 
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(TC34-3,  p.  5-1).  This  is  to  be  accomplished  through  the 
use  of  situation,  event,  and  decision  support  templates. 
These  templates  are  proposed  as  a  means  of  helping  com¬ 
manders  "visualize"  enemy  capabilities  in  a  particular 
combat  setting.  They  require  intelligence  analysts  to  make 
a  number  of  different  kinds  of  judgments.  Both  circulars 
fail,  however,  to  tell  intelligence  analysts  how  to  make 
these  judgments.  These  different  judgments  are  identified 
below  for  each  template,  in  turn.  In  addition,  limitations 
in  the  circulars  are  identified,  and  suggestions  are  offered 
for  their  improvement. 

The  situation  template  shows  how  enemy  forces 
probably  would  look  within  the  different  mobility  corridors 
under  consideration.  The  analysts  developing  the  situation 
template,  therefore,  must  use  the  terrain  and  weather  analy¬ 
ses  to  modify  the  doctrinal  templates  for  the  OPFOR  threat 
evaluated  in  step  #1.  Underlying  the  development  of  this 
template  is  a  complex  judgmental  process.  We  quote  TC30-27 
on  this  point. 

"While  the  enemy  commander  may  not  have  unlimited 
options  as  to  possible  courses  of  action,  he  will 
probably  have  enough  options  to  make  the  analyst's  job 
of  determining  probable  courses  of  action  extremely 
difficult.  Situation  templates  are  derived  based  on 
the  best  military  judgment  of  the  analyst  (p.  4-4)." 

Unfortunately,  TC30-27  provides  minimal  information  on  how 
analysts  are  to  exercise  their  "...  best  military  judgment  .. 
whe..  developing  the  situation  templates.  In  fact,  it  does 
not  even  identify  the  factors  that  are  to  be  considered 
explicitly  when  making  their  judgment. 

If  the  situation  template  is  to  reflect  military 
judgment  about  different  OPFOR  courses  of  action,  it  is  not 


enough  to  show  analysts  how  to  perform  a  terrain  and  weather 
analysis.  In  addition.  Circulars  TC30-27  and  TC34-3  should 
identify  the  factors  that  are  to  be  considered  when  making 
this  judgment,  for  the  analysis  team  will  have  to  incorporate 
judgments  about  other,  more  ambiguous  factors,  such  as  per¬ 
ceived  U.S.  force  strength  and  risk,  that  an  OPFOR  commander 
would  certainly  consider  when  selecting  a  course  of  action. 
Furthermore,  the  circulars  should  discuss  the  trade-offs 
that  OPFOR  commanders  are  likely  to  make  when  evaluating  the 
utility  of  different  courses  of  action.  Rarely  will  it  be 
true  that  one  course  of  action  is  better  than  all  others  on 
every  factor.  Enemy  commanders  will  be  forced  to  differen¬ 
tially  weight  aspects  of  their  doctrine  with  the  character¬ 
istics  of  the  situation  immediately  facing  them.  Circulars 
TC30-27  and  TC34-3,  or  supporting  documentation,  should 
indicate  what  these  trade-offs  are  likely  to  be  under  dif¬ 
ferent  terrain  and  weather  constraints,  if  the  situation 
template  is  to  accurately  represent  military  judgment  under 
different  circumstances.  Said  differently,  it  should  de¬ 
scribe  the  left  side  of  the  lens  model  in  Figure  2-1,  which 
describes  how  the  enemy  tends  to  make  judgments  of  intent. 

"The  event  template  is  a  time  and  logic  sequence 
of  enemy  tactical  indicators  or  events  which  are  keyed  to  a 
series  of  situational  templates"  (TC30-27,  pp.  1-6).  This 
is  to  be  accomplished  by  identifying,  on  the  situation  tem¬ 
plate,  "NAMED  AREAS  OF  INTEREST  (NAI's)...  along  each  route 
where  the  analyst  expects  to  see  certain  events  or  activities 
occur"  (TC30-27,  p.  4-4).  In  addition,  the  analyst  is  to 
complete  an  event  analysis  matrix  that  "...  correlates  WHAT 
IS  EXPECTED  (event/activity)  to  the  WHERE  and  WHEN  (Geographical 
coordinates  and  time)"  (TC30-27,  p.  4-6).  The  event  template 
is  to  be  a  combination  of  the  situation  template  and  the 
event  analysis  matrix. 
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Although  TC30-27  and  TC34-3  provide  no  example 
of  an  event  template,  it  appears  that  the  template  must 
identify  the  different  types  of  information  necessary  to 
confirm  or  deny  the  adoption  of  a  particular  course  of 
action.  To  accomplish  this,  the  analysts  must  make  a  series 
of  conditional  probability  judgments  analogous  to  those  rep¬ 
resented  in  Figures  2-2  and  2-3.  That  is,  they  must  say, 

"If  the  enemy  actually  took  this  particular  course  of  action, 
then  these  indicators,  and  events  (e.g.,  have  a  higher 

probability  of  being  observed  than  others."  The  word  "proba¬ 
bility"  must  be  emphasized  because  there  is  not  a  perfect 
relationship  between  indicators,  events,  and  actual  enemy 
intent.  The  enemy  will  be  expected  to  use  deception  measures. 
In  fact,  "An  integral  part  of  templating  is  the  consideration 
of  deception  events  associated  with  each  course  of  action" 
(TC30-27 ,  p.  4-11) .  As  a  result,  the  intelligence  team  will 
be  forced  to  make  conditional  probability  judgments  about 
what  indicators  and  events  they  think  are  most  indicative  of 
the  OPFOR  adoption  of  different  courses  of  action.  Neither 
TC30-27  nor  TC34-3,  provides  any  information  about  the 
probabilistic  relations  between  different  indications  and 
different  OPFOR  threats,  a  point  made  previously  when  dis¬ 
cussing  TI  zone  determination  and  evaluation. 

The  third,  and  final,  effort  in  threat  integra¬ 
tion  is  development  of  a  decision  support  template.  This 
template  "...  is  used  to  illustrate  enemy  probable  courses 
of  action  as  the  basis  for  comparing  friendly  courses  of 
action”  (TC30-27,  p.  4-13).  Described  in  this  way,  the  de¬ 
cision  support  template  "...  is  essentially  the  INTELLIGENCE 
ESTIMATE  in  graphic  format"  (TC34-3,  p.  5-10).  It  represents 
the  analysts'  most  up-to-date  estimate  about  the  relative 
likelihood  of  the  enemy's  potential  courses  of  action. 

Again,  however,  TC30-27  and  TC34-3  fail  to  indicate  how  the 
judgments  underlying  their  graphic  representation  are  to  be 
made,  or  in  fact  are  made,  by  individual  analysts.  In  terms 
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of  Figure  2-1,  they  fail  to  describe  the  right  side  of  the 
lens  model,  the  reasons  for  judgments  of  enemy  intent.  Psy¬ 
chological  research  suggests  that  this  may  result  in  decision 
support  templates  that  (1)  are  not  as  accurate  as  they  could 
be,  and  (2)  do  not  facilitate  communication  between  analysts 
and  commanders  as  much  as  they  could  otherwise. 

2.2.5  Template  Revision  -  Up  to  this  point,  there  has 
been  no  discussion  of  the  dynamic  nature  of  the  proposed  1PB 
templating  process.  The  templating  process  is  not  supposed 
to  stop  with  the  first  decision-support  template.  "They  are 
to  be  changed,  deleted,  or  redone  as  conditions  and  situations 
demand.  Used  properly,  they  provide  for  continuing  integra¬ 
tive  analysis  of  OPFOR  capabilities,  vulnerabilities  and 
courses  of  action"  (TC30-27,  p.  1-5).  This  iterative  revi¬ 
sion  process  is  critical  to  friendly  tactical  decision 
making.  Friendly  commanders  will  make  particular  decisions, 
and  take  subsequent  actions,  on  the  basis  of  intelligence 
analysts'  estimate  of  enemy  intent.  This  estimate  will  be 
revised  continuously  upon  the  arrival  of  new  information. 

The  faster  the  information  can  be  correlated  with  enemy 
intent,  the  more  time  friendly  commanders  will  have  for 
tactical  decision  making  and  action. 

Template  revision  is  represented  pictorially  in 
Figure  2-4.  Notice  that  the  decision  support  template  at 
the  end  of  one  cycle  of  the  process  is  the  situation  tem¬ 
plate  at  the  beginning  of  the  next  cycle.  The  event  tem¬ 
plates  and  event  analysis  matrices  are  used  to  convey,  as 
quickly  as  possible,  revised  estimates  about  the  likelihood 
of  different  OPFOR  courses  of  action.  These  revised  esti¬ 
mates  of  enemy  intent  are  represented  in  the  decision  sup¬ 
port  template. 

The  dynamic  nature  of  the  proposed  templating 
process  is  heavily  dependent  on  a  complex  judgment  process. 
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SCHEMATIC  REPRESENTATION  OF  THE  DYNAMIC  NATURE 
OF  THE  PROPOSED  IPB  TEMPLATING  PROCESS 


First,  intelligence  analysts  using  the  proposed  IPB  analysis 
process  must  identify  the  initial  hypotheses  regarding  pos¬ 
sible  enemy  courses  of  action  and,  if  possible,  an  initial 
estimate  of  the  most  probable  course  of  action.  As  discussed 
above,  this  requires  consideration  of  how  the  enemy  generally 
makes  judgments  of  intent,  in  addition  to  an  evaluation  of 
how  physical  terrain  and  weather  factors  favor  different 
courses  of  action. 

Second,  intelligence  analysts  using  the  proposed 
IPB  process  must  indicate  the  events  that  are  likely  to  be 
observed  for  each  course  of  action.  The  information  within 
each  event  template  represents,  in  qualitative  or  intuitive 
terms,  conditional  probabilities.  If  the  enemy  actually 
took  a  particular  course  of  action,  then  certain  events  have 
a  higher  probability  of  being  observed  than  others.  The 
information  within  all  the  event  templates  represents,  again 
in  qualitative  terms,  likelihood  ratios  that  indicate  how 
much  more  likely  certain  events  are  to  be  observed  for  one 
course  of  action  than  another.  While  these  probabilistic 
judgments  are  not  being  made  quantitatively  when  analysts 
develop  the  event  templates,  they  are  being  made  intuitively 
in  order  to  guide  data  collection  in  an  uncertain  environment. 

Third,  the  analysts  must  use  the  many  pieces  of 
potentially  fallible  data  reported  in  the  event  matrices 
(and  other  sources)  to  revise  their  initial  hypotheses  about 
the  enemy's  most  likely  course  of  action.  The  data  reported 
in  the  event  matrices  also  represent  conditional  probabili¬ 
ties,  for  certain  events  have  a  higher  probability  of  being 
observed  only  when  the  enemy  is  actually  taking  a  particular 
course  of  action.  As  a  result,  subsequent  decision  support 
templates  represent,  again  in  qualitative  terms,  the  analysts' 
most  up-to-date  estimate  about  enemy  intent,  as  revised  by 
newly  acquired  intelligence  data. 
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The  three  areas  of  judgment  inherent  in  the  re¬ 
vision  process  can  be  represented  conceptually  by  the  modi¬ 
fied  lens  model  shown  in  Figure  2-5.  The  situation  template 
indicates  the  analysts'  military  judgment  about  the  likeli¬ 
hood  of  different  enemy  courses  of  action.  The  event  tem¬ 
plates  indicate  events  the  analysts  expect  to  see  in  support 
of  each  course  of  action.  The  event  analysis  matrices  indi¬ 
cate  the  events  subsequently  reported  in  intelligence  data. 
The  events  "seen"  are  matched  with  those  "expected"  in  order 
to  develop  a  revised  estimate  of  enemy  intent,  which  is  rep¬ 
resented  in  the  decision  support  template.  The  decision 
support  template  is  now  the  situation  template  for  the  next 
iteration. 


There  are  different  ways  to  represent  quantita¬ 
tively  the  three  areas  of  judgment  inherent  in  the  template 
revision  process.  These  procedures  will  be  considered  in 
some  detail  in  the  next  section  when  we  consider  different 
decision  aids  for  supplementing,  and  thereby  improving  the 
proposed  IPB  analysis  process.  Suffice  it  to  say  for  now, 
that  these  procedures  often  rely  on  the  principle  of  divide 
and  conquer.  First,  the  total  problem  is  divided  into  a 
series  of  structurally  related  parts.  The  intelligence 
analyst  is  asked  to  evaluate  each  OPFOR  alternative  under 
consideration  for  each  of  the  independent,  simpler  components 
that  comprise  the  larger  intelligence  problem.  Then,  compu¬ 
terized  judgment  aids  combine  all  the  judgments  in  order  to 
provide  an  overall  evaluation  of  each  alternative.  Many  of 
the  aids  also  provide  sensitivity  analysis,  thereby  permitting 
intelligence  analysts  to  observe  the  effects  of  changing 
their  judgments  on  the  overall  score  for  each  alternative. 
Thus,  decision-analytic  techniques  and  computerized  aids 
"conquer"  the  global  decision  problem  by  providing  an  analyt¬ 
ical  means  of  expanding  the  intelligence  analyst's  cognitive 
skills. 
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2 . 3  Summary 

The  purpose  of  IPB  is  to  improve  procedures  for  corre¬ 
lating  data  with  enemy  intent.  This  purpose  is  to  be  accom¬ 
plished  by  templating,  an  analytical  tool  designed  to  help 
analysts  and  commanders  visualize  the  battlefield.  While 
important,  this  advance  does  not  go  far  enough,  for  it  fails 
to  make  explicit  how  input,  new  intelligence  data  (X^) ,  is 
to  be  converted  into  output,  templates.  This  translation 
process  necessarily  depends  on  judgment.  However,  Circulars 
TC30-27  and  TC34-3  provide  practically  no  information  about 
how  these  judgments  are  to  be  made  by  analysts. 

Section  2.0  described  the  necessary  dependence  of  IPB 
on  human  judgment.  In  particular,  it  provided  a  conceptual 
overview  of  the  causal  inference  problem  inherent  in  intelli¬ 
gence  analysis,  and  identified  the  different  types  of  judg¬ 
ment  inherent  in  each  of  the  five  steps  in  the  IPB  analysis 
process  proposed  in  Circulars  TC30-27  and  TC34-3.  The  gen¬ 
eral  conclusion  was  that  except  for  terrain  analysis  (Step  3) 
and  weather  analysis  (Step  4),  these  circulars  fail  to  tell 
intelligence  analysts  how  they  are  to  make  the  judgments 
necessary  to  implement  the  proposed  IPB  analysis  process. 

The  lack  of,  and  need  for,  procedures  for  helping  analysts 
make  the  judgments  in  the  other  three  steps  of  the  proposed 
IPB  analysis  process  were  discussed  at  length. 

It  is  important  to  note  that  the  proposed  IPB  analysis 
process  is  a  dynamic  one,  for  the  decision-support  template 
at  the  end  of  one  cycle  of  the  process  is  the  situation 
template  at  the  beginning  of  the  next  cycle.  Three  areas  of 
judgment  are  inherent  in  the  revision  process.  These  areas 
of  judgment  were  represented  conceptually  within  the  causal 
inference  model  presented  in  Section  2.0.  Analytical  pro¬ 
cedures  and  computerized  judgment  aids  for  assisting  analysts 
tasked  with  making  these,  and  the  other  judgments  in  the 
proposed  IPB  analysis  process,  are  described  in  Section  3.0. 
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3.0  CAUSAL  INFERENCE  RESEARCH  FINDINGS  AND 
COMPUTER-BASED  JUDGMENT  AIDS  APPLICABLE  TO 
INTELLIGENCE  PREPARATION  OF  THE  BATTLEFIELD 

The  previous  section  of  this  report  described  the  dif¬ 
ferent  types  of  judgments  inherent  in  the  IPB  analysis  pro¬ 
cess,  as  proposed  in  TC30-27  and  TC34-3.  This  section 
describes  the  results  of  research  studying  how  well  people 
make  these  judgments.  The  research  strongly  suggests  that 
people's  ability  to  make  these  judgments  can  be  improved 
substantially  by  giving  them  access  to  computerized  decision 
aids.  These  aids,  therefore,  represent  an  important  means 
of  improving  the  proposed  IPB  analysis  process. 

This  section  is  divided  into  two  parts.  The  first  part 
describes  causal  inference  research  findings  and  computerized 
judgment  aids  particularly  relevant  to  the  development  of 
the  initial  decision  support  template.  The  second  part 
describes  research  and  aids  particularly  relevant  to  the 
revision  of  decision  support  templates  on  the  basis  of  newly 
collected  data  represented  in  the  event  template  and  event 
analysis  matrix.  The  difference  in  emphasis  is  used  here  to 
indicate  the  strengths  of  each  class  of  judgment  aids; 
however ,  both  types  of  aids  can  be  used  in  initial  devel¬ 
opment  and  subsequent  revision  of  decision  support  templates. 

Research  and  Aids  Applicable  to  the  Development  of  the 
Initial  Decision  Support  Template 

As  discussed  in  Section  2.0,  the  decision  support  tem¬ 
plate  must  identify  the  different  possible  enemy  courses  of 
action  and  the  most  likely  enemy  choice.  To  make  this 
latter  judgment,  the  analysis  team  must  integrate  individual 
judgments  of  OPFOR  doctrine  with  those  pertaining  to  the 


military  aspects  of  terrain  and  weather.  In  addition,  the 
analysis  team  must  (1)  incorporate  judgments  about  other, 
more  ambiguous  factors  (or  attributes) ,  such  as  perceived 
U.S.  force  strength  and  risk,  that  an  OPFOR  commander  would 
certainly  consider  when  selecting  a  course  of  action,  and 
(2)  estimate  the  trade-offs  that  OPFOR  commanders  are  likely 
to  make  when  evaluating  the  utility  of  different  courses  of 
action.  For  rarely  will  it  be  true  that  one  course  of  ac¬ 
tion  is  better  than  all  others  on  every  value-relevant  at¬ 
tribute.  Enemy  commanders  will  be  forced  to  differentially 
weight  aspects  of  their  doctrine  with  the  characteristics  of 
the  situation  immediately  facing  them.  By  accurately  pre¬ 
dicting  these  trade-offs,  and  by  representing  them  in  the 
first  decision  support  template,  intelligence  analysts  can 
provide  friendly  commanders  with  time  for  efficient  tactical 
decision  planning. 

This  section  reviews  laboratory  research  on  causal  in¬ 
ference  conducted  within  the  Brunswickian  framework  repre¬ 
sented  in  Figures  2-1,  2-2,  and  2-3.  In  particular,  Section 
3.1.1  reviews  research  investigating  the  accuracy  of  the 
judgments  of  individual  participants  and  the  obtained  level 
of  interpersonal  agreement  in  different  settings  where  par¬ 
ticipants  had  no  access  to  judgment  aids.  Section  3.1.2 
reviews  research  findings  regarding  the  obtained  levels  of 
individual  accuracy  and  interpersonal  agreement  when  parti¬ 
cipants  had  access  to  judgment  aids.  Section  3.1.3  describes 
a  judgment  aid  designed  to  assist  Staff  Intelligence  Officers 
(G2/S2s)  in  looking  at  the  battlefield  from  the  perspective 
of  enemy  commanders  prior  to  their  decision  to  adopt  a 
course  of  action;  therefore,  it  provides  an  aid  to  the  de¬ 
velopment  of  the  initial  decision  support  template. 

3.1.1  Research  findings:  without  judgment  aids  - 
Considerable  psychological  research  on  the  accuracy  of  ex¬ 
pert  judgment  and  the  level  of  interpersonal  agreement  has 
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been  conducted  within  the  conceptual  framework  of  Brunswik's 
Lens  Model  (1952,  1956).  This  research  has  been  conducted, 
for  example,  with  clinical  psychologists  (e.g.,  Goldberg, 
1970;  Hammond,  Hursch,  and  Todd,  1964;  Meehl,  1954),  stock¬ 
brokers  (Ebert  and  Kruse,  1978;  Slovic,  Fleissner,  and 
Bauman,  1972),  radiologists  (Hoffman,  Slovic,  and  Rorer, 
1968) ,  histologists  (Einhorn,  1974)  and  highway  engineers 
(Adelman  and  Mumpower,  1979).  In  situations  with  criterion 
data,  the  judgmental  accuracy  of  these  experts  often  has 
been  found  to  be  lower  than  expected  by  the  researchers.  In 
addition,  interpersonal  agreement  has  been  found  to  be  sur¬ 
prisingly  low. 

Research  by  Ebbesen  and  Konecni  (1975)  with  ju¬ 
rists  and  Phelps  and  Shanteau  (1977)  with  livestock  experts 
indicates  that  the  level  of  judgmental  accuracy  and  inter¬ 
personal  agreement  can  be  a  direct  function  of  the  charac¬ 
teristics  of  the  judgmental  task  facing  the  experts.  For 
example,  both  found  that  the  degree  of  correlation  between 
attributes  will  affect  the  number  of  attributes  apparently 
used  by  experts.''"  This  general  conclusion  is  further  sup¬ 
ported  by  controlled  laboratory  research  which  has  shown 
that  (1)  judgmental  accuracy  and  agreement  can  be  increased 
or  decreased  by  manipulating  characteristics  of  the  task  and 
(2)  that  relatively  high  levels  of  accuracy  and  agreement 
can  be  maintained  under  conditions  that  normally  prevent  it 
by  using  judgment  aids  that  provide  persons  with  information 
about  task  characteristics. 


^The  word  "apparently"  is  used  because  no  statistical  analy¬ 
sis  procedure  can  identify  how  a  person  actually  used  the 
information;  all  it  can  do  is  identify  the  model  of  infor¬ 
mation  use  that  best  predicts  the  person's  judgment.  See 
Hoffman  (1960)  for  an  early,  yet  detailed  discussion  of 
this  point. 
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Causal  inference  tasks  can  be  characterized  in 
terms  of  their  formal  characteristics.  Formal  task  charac¬ 
teristics  refer  to  the  statistical  relationships  among  task 
variables.  Research  on  the  following  two  task  characteris¬ 
tics  is  of  particular  relevance  to  effective  intelligence 
analysis:  (a)  the  predictability  of  individual  pieces  of 

information  (called  cues  or  attributes),  and  (b)  the  total 

2 

predictability  of  all  the  cues  in  the  inference  task. 

In  the  standard  procedure  for  studying  causal  in¬ 
ference,  the  participant  is  shown  a  set  of  "cases"  on  which 
to  make  judgments.  In  terms  of  Figure  2-1,  each  case  is  de¬ 
scribed  by  values  on  a  set  of  cues  (or  attributes)  (XJ  that 
participants  use  to  make  a  prediction  (Y  )  of  the  criterion 

or  "cause"  (Y  ) .  Formal  task  characteristics  determine  the 
e 

relationship  between  the  criterion  and  the  cue  values. 
Participants  are  shown  the  correct  answer  after  each  case, 
so  they  can  learn  how  to  make  accurate  causal  inferences. 

After  learning  has  occurred,  judgmental  accuracy  (or  "achieve¬ 
ment")  is  measured  statistically. 

Hursch,  Hammond,  and  Hursch  (1964)  and  Tucker 
(1964)  have  shown  that  achievement  can  be  partitioned  into 
statistically  independent  components.  Under  conditions  in 
which  the  criterion  values  can  be  expressed  as  a  linear 
function  of  the  attribute  values,  Tucker's  formula  can  be 
stated  as  follows: 

£a  =  2%Ke  (1) 

2 

The  words  "cue"  and  "attribute"  are  used  interchangeably 
throughout  this  section  to  provide  continuity  between  re¬ 
searchers  within  the  Brunswikian  framework  who  use  the  word 
"cue,"  and  decision  analysts  developing  judgment  aids  who 
use  the  word  "attribute."  The  word  "cue"  is  more  appropri¬ 
ate  in  this  section  because  information  (X^)  is  a  "cue"  to 
the  correct  criterion  (Ye) ,  not  an  attribute  of  it. 


In  this  formulation,  r  is  the  correlation  between  criterion 

-a 

values  and  the  participant's  judgments  of  these  values; 
hence,  r  provides  a  measure  of  achievement.  G  is  the  cor- 
relation- between  the  linear  variance  in  the  task  system  and 
the  linear  variance  in  the  participant's  judgment  system. 
Since  a  high  G  indicates  that  the  properties  of  the  partici¬ 
pant's  judgment  system  match  the  properties  of  the  task  sys¬ 
tem,  G  provides  a  measure  of  the  participant's  knowledge  of 
task  properties.  Rg  is  the  linear  multiple  correlation  be¬ 
tween  the  attribute- values  and  the  participant's  judgments. 
Since  a  high  Rg  indicates  that  participants  are  making  con¬ 
sistent  judgments  based  on  whatever  (not  necessarily  cor¬ 
rect)  knowledge  of  the  task  they  have  acquired,  it  provides 
a  measure  of  "cognitive  control"  (Hammond  and  Summers,  1972) 
or  the  ability  of  participants  to  consistently  make  judgments 
in  the  way  they  want  to  do  so.  is  the  linear  multiple 

correlation  between  the  attr ibute- and  criterion  values;  it 

is  the  task's  counteroart  of  R  and,  therefore,  is  a  measure 

— s 

of  task  consistency  or  predictability. 

It  is  important  to  note  that  Rg  is  statistically 
independent  of  G.  This  indicates  that  even  if  G  reaches 
unity  (indicating  perfect  knowledge) ,  if  Rg  were  less  than 
unity  (indicating  imperfect  control) ,  achievement  (r  ) 
would  be  less  than  the  level  of  task  predictability  TR  ) . 
Conversely,  the  same  results  could  occur  if  Rg  equalled"  1.0, 
but  G  were  less  than  1.0.  Therefore,  two  persons  can  have 

identical  achievement  indices  (r  )  for  different  reasons: 

—a 

one  because  of  perfect  knowledoe  (G  =  1.0)  but  imperfect 
control  (Rg  <  1.0),  the  other  because  of  perfect  control 
(R,  =  1.0)*" but  imperfect  knowledge  (G  <  1.0)  This  distinc¬ 
tion  is  critical  when  discussing  the  effects  of  formal  task 
characteristics  on  the  accuracy  of  causal  inference. 

Variation  in  the  predictability  of  individual 
cues,  for  example,  affects  achievement  by  affecting  both  the 
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acquisition  and  the  utilization  of  knowledge  about  the 

causal  environment.  For  example,  Uhl  (1963)  investigated 

the  effects  of  varying  the  predictability  of  individual  cue 

validity  by  constructing  7  three-cue  tasks  ranging  in 

individual  cue  predictability  from  one  extreme  where  all  the 

cues  were  equally  predictable  (i.e.,  r  .  =  r~  =  r  where 

r  .  is  the  correlation  between  the  cue- and  tEe  criterion) , 

— ei 

to  the  extreme  where  only  one  cue  predicted  the  criterion. 

In  all  tasks  the  cues  were  interval  in  nature,  linear  and 

orthogonal.  And  in  all  cases,  overall  task  predictability 

was  unity  (i.e.,  =  1.0).  In  these  tasks,  achievement 

(r  )  was  poorest  in  tasks  where  individual  cues  had  equal 
— a. 

predictability.  Lower  achievement  in  such  tasks  resulted 
from  both  lower  knowledge  (G)  and  lower  cognitive  control 
(R  )  than  in  the  tasks  with  a  larger  disparity  in  the  pre¬ 
dictability  of  individual  attributes.  Dudycha  and  Naylor 
(1966)  found  similar  results.  These  results  are  important 
to  intelligence  analysts  because,  according  to  the  report  b 


Bowen,  et  al.  (1975)  reviewing  Tactical  Order  of  Battle,  in¬ 


telligence  analysts  have  to  consider  many  information  ele¬ 
ments  of  relatively  equal  importance. 

Variation  in  overall  task  predictability  (Rc) 
affects  achievement  (r  )  by  leading  to  variation  in  cogni- 

"  u 

tive  control  (Rg) ,  not- knowledge  (G) .  For  example,  Uhl 
(1966)  investigated  the  effects  of  task  predictability  on 
achievement  by  constructing  four  three-attribute  tasks 
ranging  in  predictabilities  of  1.0,  0.67,  0.33,  and  0.00. 
The  attributes  were  orthogonal  and  linear;  individual  at¬ 
tribute  predictability  (r^)  was  0.775,  -0.225,  and  0.000, 
respectively.  Uhl  (1966 )— Found  that  there  was  a  positive 

linear  function  between  task  achievement  and  task  predict- 

2 

ability;  i.e.,  r  decreased  as  R  decreased.  The  cause  of 

1  Q  G 

lower  achievement,  however,  was  not  lower  knowledge  (G) , 
but  lower  cognitive  control  (R  ) ,  for  there  was  a  positive 
linear  function  between  task  predictability  and  cognitive 
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2  2 

control;  i.e.,  as  R  decreased,  R  decreased.  Furthermore, 

^  2  £  2 
the  match  between  r"?  and  R_  was  quite  high,  except  for  R_ 

'  --  “  C  _  ^  ”S  c 

*  0.00;  specifically,  R”  was  0.90,  0.66,  0.39,  and  0.29  Tor 
-2  — 

Rc  of  1.0,  0.67,  0.33,  and  0.00,  respectively.  Therefore, 
persons  not  only  matched  the  predictability  of  individual 
attributes,  they  matched  overall  task  predictability  (or 
inversely,  randomness)  as  well.  Dudycha  and  Naylor  (1966) 
and  Schmitt,  Coyle,  and  King  (1976)  found  similar  results. 
This  finding  is  of  particular  importance  to  intelligence 
analysts  because  the  inherently  low  level  of  overall  pre¬ 
dictability  in  some  intelligence  tasks  suggests  that  ana¬ 
lysts  may  perform  suboptimally  because  of  cognitive  in- 
ability  to  utilize  their  knowledge  of  enemy  activities  in 
a  highly  consistent  fashion. 

Controlled  laboratory  research  has  shown  that 
formal  task  characteristics  also  affect  interpersonal  agree¬ 
ment  (see  Brehmer,  1976,  for  an  extensive  review  of  this 
literature) .  The  procedure  for  this  research  has  two  stages 
a  training  stage  in  which  each  participant  learns  to  make 
causal  inferences  in  a  certain  way,  and  a  conflict  stage  in 
which  participants  make  judgments  jointly.  The  lens  model 
equation  is  again  used,  but  now  r  is  the  correlation  be- 
tween  the  judgments  of  the  two  participants;  hence,  r^ 
represents  the  overall  level  of  agreement  between  the- par¬ 
ticipants,  G  represents  the  similarity  in  their  judgmental 
policies  (i.e.,  the  way  they  make  causal  inferences),  and 
Rgl  and  Rg2  represent  their  levels  of  judgmental  consis¬ 
tency. 


Two  task  characteristics  affecting  interpersonal 
agreement  that  are  of  particular  relevance  to  intelligence 
analysis  are  overall  task  predictability  and  cue  intercorre¬ 
lations.  Regarding  the  former,  the  lower  the  level  of  task 
predictability,  the  lower  the  level  of  interpersonal  under¬ 
standing  and  agreement.  The  cause  of  this  result  is  lower 


judgmental  consistency  (or  cognitive  control)  on  the  part  of 
each  participant,  not  policy  similarity.  This  occurs  because 
people  try  to  predict  the  randomness  (i.e.,  uncertainty)  in 
the  causal  inference  task.  As  a  result,  they  make  it  diffi¬ 
cult  for  the  other  person  to  understand  how  they  make  their 
judgments,  and  thereby,  how  to  resolve  the  disagreement.  In 
fact,  Brehmer  (1972)  found  that  low  levels  of  task  predicta¬ 
bility  could  even  cause  people  who  began  the  conflict  stage 
in  perfect  agreement  to  finish  it  in  disagreement  because  of 
judgmental  inconsistency.  Therefore,  low  overall  task  pre¬ 
dictability,  the  same  formal  task  property  that  causes  sub- 
optimal  judgmental  accuracy,  also  causes  interpersonal  dis- 
agreements .  Research  indicates  that  computerized  judgment 
aids  improve  the  levels  of  both  accuracy  and  agreement. 


Cue  intercorrelations  also  introduce  certain 
constraints  on  the  relationship  among  judgmental  accuracy, 
interpersonal  agreement,  and  policy  similarity.  If  the  cues 
have  a  high  positive  correlation,  for  example,  then  a  person 
can  rely  totally  on  an  invalid  cue  and  still  be  accurate 
because  it  does  not  matter  which  cue  is  used  to  predict  the 
criterion.  Furthermore,  that  person's  judgments  will  agree 
with  those  of  another  person  who  is,  in  fact,  relying  on  the 
valid  cue.  If  the  situation  should  change,  however,  such 
that  the  cues  no  longer  have  a  high  positive  correlation, 
then  the  two  persons  will  be  in  strong  diagreement.  Fur¬ 
thermore,  they  will  have  great  difficulty  resolving  their 
dispute  because  both  will  point  to  their  recent  successes, 
although  only  one  of  the  people  actually  knows  which  cue  is 
the  valid  indicator. 


3.1.2  Research  findings:  with  judgment  aids  -  One 
question  given  considerable  consideration  by  researchers 
studying  causal  inference  research  has  been,  how  can  achieve¬ 
ment  or  accuracy  be  improved?  The  traditional  answer,  con¬ 
sistent  with  a  stimulus-response-reinforcement  orientation. 
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has  been  to  provide  the  learner  with  outcome  feedback,  that 
is,  the  correct  answer  after  each  learning  trial.  This  is 
conceptually  the  same  answer  given  when  intelligence  ana¬ 
lysts  are  forced  to  "learn  by  experience."  Yet,  as  Todd  and 
Hammond  (1965,  pp.  429-430)  point  out, 

"Outcome  feedback  may  be  appropriate  when  the 
learning  task  is  sufficiently  simple  that  the  subject  can 
associate  specific  responses  with  specific  stimulus  configu¬ 
rations,  .  .  .  [but]  few  psychologists  would  seriously  en¬ 
tertain  the  idea  that  such  specific  associations  are  what  is 
being  learned  in  multiple-cue  probability  tasks  in  which 
very  large  numbers  of  such  associations  are  typically  re¬ 
quired.  More  probably,  it  is  the  relations  between  cues  and 
criteria  which  are  learned.  Outcome  feedback,  however,  does 
not  appear  to  be  appropriate  for  learning  probability  rela¬ 
tions,  because  it  yields  information  which  is  restricted  to 
a  comparison  of  end  results — the  comparison  of  the  response 
with  the  correct  answer.  .  .  .  Feedback  which  directly  in¬ 
forms  the  £  about  the  relation  of  his  cue  utilizations 
(rg^)  should  make  it  possible  for  the  S  to  adjust  his  cue 
utilizations  in  the  direction  of  the  cue  validities." 

Feedback  that  permits  persons  to  compare  the 
formal  properties  of  their  judgmental  system  (right-hand 
side  of  Figure  2-1)  with  those  of  the  task  system  (left-hand 
side  of  Figure  2-1)  has  been  called  "lens  model  feedback" 

(Todd  and  Hammond,  1965,  p.  431),  or  more  recently,  "cognitive 
feedback"  (Hammond,  Stewart,  Brehmer,  and  Steinmann,  1975, 
p.  293) .  Computer  graphics  devices  have  been  used  widely  as 
decision  aids  for  presenting  cognitive  feedback.  Hammond 
(1971),  for  example,  has  shown  how  computer  graphics  can  be 
used  to  provide  participants  with  a  pictorial  and  quantita¬ 
tive  representation  of  achievement  in  terms  of  the  lens 
model  and  lens  model  equation,  respectively.  Figure  3-1 
provides  an  example  of  such  a  pictorial,  quantitative 
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EXAMPLES  OF  COGNITIVE  FEEDBACK  DISPLAYS 
(From  Hammond  and  Boyle,  1971) 


display.  Cognitive  feedback  provided  by  such  displays  has 
been  shown  to  lead  to  both  a  faster  rate  of  learning  and  a 
higher  level  of  achievement  (r^)  than  outcome  feedback  in 
tasks  that  varied  the  (a)  disparity  of  the  individual  pre¬ 
dictabilities  of  the  attributes  (Lindell,  1976),  and  (b)  the 
overall  level  of  task  predictability  (Schmitt,  Coyle,  and 
King,  1976) .  While  this  finding  is  due  in  some  part  to 
greater  knowledge  acquisition  (G)  with  cognitive  than  out¬ 
come  feedback,  it  is  primarily  due  to  higher  cognitive  con¬ 
trol  with  cognitive  feedback.  This  finding  strongly  sup¬ 
ports  our  position  that  computerized  judgment  aids  will  im¬ 
prove  the  skills  and  judgmental  accuracy  of  experienced  in- 
telligence  analysts  using  IPB  by  greatly  increasing  their 
ability  to  implement  their  knowledge. 


Laboratory  research  by  Hammond  and  Brehmer 
(1973)  and  by  Summers,  Taliaferro,  and  Fletcher  (1970a)  has 
found  that  computerized  judgment  aids  that  quantitatively 

describe  how  people  combine  information  on  multiple  cues  i 

into  an  overall  judgment  greatly  enhances  interpersonal 
understanding.  The  reason  for  this  is  that  decision  aids 
overcome  the  inaccuracy  and  inconsistency  of  verbal  self- 
reports.  Verbal  reports  are  often  inaccurate  because 
people  inaccurately  estimate  the  weight  they  place  on  vari¬ 
ous  attributes.  Research  by  Cook  and  Stewart  (1975)  and 
Summers,  Taliaferro,  and  Fletcher  (1970b)  indicates  that 
people  sometimes  underestimate  the  weight  they  place  on  im¬ 
portant  factors  and  overestimate  the  weight  they  place  on 
unimportant  factors  when  compared  to  a  quantitative  analysis 

of  their  judgments.  To  put  it  bluntly,  people  are  sometimes  jr 

unaware  of  their  own  judgment  policy;  therefore,  it  is  not 
surprising  that  they  inaccurately  describe  it. 


Decision-analytic  techniques  and  computerized 
judgment  aids  can  be  used  to  describe  explicitly  (1)  how 
different  persons  make  their  causal  inference  judgments, 
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(2)  the  intercorrelations  perceived  by  different  persons, 
and  (3)  how  attribute  intercorrelations  can  affect  judg¬ 
mental  accuracy  and  interpersonal  agreement.  Such  capabili¬ 
ties,  therefore,  provide  an  explicit  means  of  overcoming 
formal  task  properties  that  research  (e.g.,  Brehmer,  1974; 
Hammond  and  Mumpower,  1974)  indicates  can  cause  interper¬ 
sonal  disagreements.  Furthermore,  such  capabilities  have 
been  used  to  resolve  actual  conflicts.  (See  Hammond,  Rohr- 
baugh,  Mumpower,  and  Adelman,  1977,  for  a  review.)  For  ex¬ 
ample,  Hammond  and  Adelman  (1976)  helped  the  Denver  City 
Council  resolve  a  conflict  over  the  type  of  handgun  bullet 
to  be  used  by  the  Denver  Police  Department  by  showing  that 
there  was  not  a  perfect  relationship  between  a  bullet's 
stopping  effectiveness  and  its  severity,  as  was  perceived  by 
many  parties  to  the  conflict.  Hammond,  Rohrbaugh,  Mumpower, 
and  Adelman  (1977),  and  Kelly  (1979)  provide  reviews  de¬ 
scribing  how  judgment  aids  have  been  used  for  different 
substantive  problems. 

3.1.3  A  judgment  aid  for  IPB  -  Multi-attribute  utility 
assessment  (MAUA)  techniques  and  judgment  aids  can  be  used 
to  help  intelligence  analysts  develop  the  initial  decision 
support  template.  They  provide  (1)  a  logically  defensible 
conceptual  structure  for  structuring  the  factors  used  in 
estimating  enemy  intent,  (2)  techniques  for  estimating  the 
necessary  trade-offs  inherent  in  estimating  enemy  intent, 

(3)  an  analytical  procedure  for  combining  the  multiple 
trade-off  judgments  into  an  overall  assessment  of  the  like¬ 
lihood  of  different  enemy  courses  of  actions,  (4)  a  means  of 
systematically  investigating  the  implications  of  differences 
of  opinion  in  judgment,  and  (5)  a  means  of  conveying  all 
this  information  pictorially,  thus  providing  an  important 
adjunct  to  the  proposed  pictorial  format  of  the  decision 
support  template.  As  a  result,  a  computerized  judgment  aid 
called  ENCOA  (Enemy  Courses  of  Action)  has  been  developed  to 
assist  Staff  Intelligence  Officers  (G2/S2s)  develop  the 
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initial  decision  support  template;  see  Patterson  and  Phelps 
(1980)  for  a  complete  description. 

The  utilization  of  ENCOA,  like  all  MAUA-based 
decision  aids,  requires  five  basic  steps.  First  one  de¬ 
velops  a  hierarchy  of  attributes  (or  criteria)  that  struc¬ 
ture  the  process  of  evaluating  the  different  alternatives. 
Second,  one  scores  each  of  the  alternatives  on  each  of  the 
attributes  at  the  bottom  of  the  hierarchy.  Third,  one  spe¬ 
cifies  the  relative  importance  (or  weight)  of  the  multiple 
attributes  within  each  level  of  the  hierarchy.  Fourth,  one 
combines  the  scores  (Step  #2)  and  relative  weights  (Step  #3) 
to  obtain  an  overall  value  for  each  alternative.  Fifth,  one 
performs  sensitivity  analyses  to  determine  what  conditions 
will  change  the  conclusion  of  the  MAUA  effort.  Each  step  is 
considered  below  at  both  a  general  level  and  for  ENCOA  in 
particular . 

A  MAUA  model  is  hierarchical  in  nature,  starting 
with  the  specified  top-level  factor  for  which  an  overall 
score  is  desired.  This  factor  is  successively  decomposed 
into  subfactors  in  descending  levels  of  the  hierarchy  such 
that  each  successive  level  is  more  specific  than  the  one 
preceding.  At  the  lowest  level  of  the  hierarchy  are  pre¬ 
dictable  or  highly  observable  characteristics  of  the  alter¬ 
natives  under  evaluation. 

Figure  3-2  shows  the  MAUA  hierarchy  in  FNCOA  for 
assessing  the  overall  utility  (or  value)  of  the  different 
courses  of  action  open  to  OPFOR  commanders.  There  are  five 
major  factors  for  evaluating  the  overall  utility  of  each  al¬ 
ternative:  Terrain,  U.S.  Forces,  OPFOR  Forces,  Weather,  and 
Risk.  Each  of  these  higher  level  factors  is  decomposed  into 
more  observable  subfactors.  Figure  3-3  provides  the  defini¬ 
tions  for  these  lower  level  subfactors.  These  subfactors 
are  defined  in  a  manner  that  permits  intelligence  analysts 
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DEFINITION  OF  ENCOA  SUBFACTORS 


to  score  each  alternative  on  each  subfactor,  the  second  step 
in  using  ENCOA. 

ENCOA  uses  a  relative  scoring  system.  The  best 
OPFOR  course  of  action  on  each  subfactor  is  given  a  score  of 
100,  the  worst  OPFOR  course  of  action  on  that  subfactor  is 
given  a  0,  and  other  courses  of  action  are  given  intermediate 
values  between  100  and  0  relative  to  the  best  and  worst  al¬ 
ternatives,  respectively.  A  relative  scoring  system  is 
used  instead  of  an  absolute  scoring  system  because  of  the 
difficulty,  if  not  the  impossibility,  of  trying  to  define  a 
true  zero  level  on  each  subfactor.  An  absolute  scoring  sys¬ 
tem  necessitates  defining  a  true  zero  level  of  performance 
and  then  scoring  systems  proportional  to  how  far  they  exceed 
that  zero  level;  a  "relative"  scoring  system  arbitrarily  se¬ 
lects  the  least  desirable  outcome  on  each  criterion  as  a 
relative  zero,  and  then  scores  each  of  the  other  systems 
proportional  to  the  magnitude  of  the  difference  between  that 
system  and  the  one  with  the  lowest  score.  A  relative  scor¬ 
ing  system  is  used  in  ENCOA  because  of  the  difficulty  in  de¬ 
fining  a  true  zero  level  for  the  attributes.  As  a  result, 
some  caution  is  required  in  interpreting  the  numerical 
scores,  for  while  one  can  make  relative  comparisons  (e.g., 
better  or  worse) ,  one  cannot  make  absolute  comparisons 
(e.g.,  good  or  bad). 

The  third  step  in  assessing  total  utility  is  to 
specify  the  relative  importance  (or  weight)  of  the  attributes 
within  the  hierarchy.  The  factors  on  each  level  are  com¬ 
pared,  by  proceeding  from  bottom  to  top,  to  determine  the 
relative  importance  of  the  range  of  variation  across  the  op¬ 
tions.  Since  the  options  are  scored  on  a  relative  scale, 
the  weights  represent  the  importance  of  the  range  between 
the  best  and  worst  options  on  each  attribute  at  that  level 
of  the  hierarchy.  For  example,  if  the  difference  between  the 
OPFOR  courses  of  action  on  fields  of  fire  is  the  most  important 
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difference  among  any  of  the  terrain  subfactors,  it  is  arbi¬ 
trarily  given  a  weight  of  100.  The  intelligence  analyst 
then  must  assign  a  relative  weight  to  the  second  most  impor¬ 
tant  difference  among  the  terrain  subfactors,  the  third  most 
important,  and  so  on,  each  weight  reflecting  the  percentage 
worth  of  the  difference  under  consideration  to  that  judged 
most  important  and  assigned  a  weight  of  100.  After  weight¬ 
ing  all  the  terrain  subfactors,  the  analysts  weight  all  the 
subfactors  within  each  of  the  other  four  major  factors  or 
categories. 


After  all  the  above  scores  and  weights  have  been 
entered  by  the  user,  the  final  judgmental  operation  required 
is  to  assign  importance  weights  to  each  of  the  five  cate¬ 
gories.  To  accomplish  this,  a  set  of  five  factors  consisting 
of  the  top-weighted  factor  from  each  category  is  presented. 
Again,  the  intelligence  analyst  is  required  to  judge  the 
relative  importance  of  the  magnitude  of  the  difference  be¬ 
tween  the  courses  of  action  scored  poorest  and  best  on  each 
of  these  factors  and  to  rank  and  weight  these  differences 
just  as  before.  This  operation  has  the  effect  of  adjusting 
factor  weights  by  the  importance  weighting  of  the  category 
of  which  each  is  a  part.  If,  for  example,  the  four  factors 
under  weather  had  been  assigned  weights  of  100,  80,  50,  and 
30  on  the  basis  of  their  relative  importance  within  the 
weather  category,  and  the  weight  then  assigned  the  weather 
category  was  50,  the  adjusted  weights  for  the  weather 
factors  would  become  50,  40,  25,  and  15. 

As  Kibler  et  al.  (1978)  discuss  at  length,  the 
process  of  assigning  scores  to  courses  of  action  (i.e.,  spe¬ 
cifying  the  one  which  is  best  on  a  factor,  worst,  and  inter¬ 
mediate)  is  found  by  most  people  to  be  an  easy  task.  The 
process  of  assigning  importance  weights  to  magnitudes  of 
difference  is,  however,  initially  an  unfamiliar  way  of 
thinking  for  most.  Unless  users  of  the  decision  aid  are 
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carefully  briefed  and  prompted  in  early  trials,  there  is  a 
tendency  for  many  to  slip  into  the  conventional  (but,  in 
this  context,  erroneous)  pattern  of  assigning  weights  to  the 
generally  perceived  importance  of  a  factor,  rather  than  to 
the  importance  of  the  difference  in  value  between  the  ex¬ 
treme  courses  of  action  on  a  factor.  With  practice,  the 
correct  frame  of  reference  for  judging  importance  weights 
becomes  routine.  Even  then,  however,  there  remains  an  un¬ 
certainty  band  around  the  importance  weights  entered  into 
the  model.  The  individual  may  have  entered  a  weight  of, 
say,  70  for  a  factor  but  was  really  uncertain  as  to  whether 
that  value  might  just  as  well  have  been  60  or  80.  When  such 
uncertainty  enters  the  picture  (as  is  usually  the  case) ,  it 
is  of  great  importance  to  know  whether  variation  of  the 
judgmental  inputs  within  the  decision  maker's  band  of  error 
would  shift  the  indicated  course  of  action  selection  from 
one  option  to  another — a  matter  of  sensitivity  testing. 

In  the  fourth  step,  one  combines  the  scores  of 
step  two  and  the  relative  weights  of  step  three  to  obtain  an 
overall  utility  value  for  each  alternative.  The  utility 
values  are  obtained  by  multiplying  the  bottom-level  value 
score  of  each  option  by  its  relative  weight  to  obtain  a 
score  for  the  option  at  the  next  level.  This  procedure  con¬ 
tinues  until  each  option  receives  an  overall  utility  score 
at  the  top  of  the  hierarchy.  Since  the  calculations  are 
simple  arithmetic,  they  are  readily  performed  by  ENCOA.  As¬ 
suming  that  the  scores  and  weights  are  valid  and  that  the 
model  captures  the  salient  factors  relevant  to  the  situa¬ 
tion,  the  course  of  action  yielding  the  highest  weighted 
score  represents  the  most  likely  enemy  course  of  action  in 
the  opinion  of  the  participating  intelligence  analysts. 

The  fifth  and  final  step  in  utilizing  MAUA 
through  computerized  decision  aids  like  ENCOA  is  sensitivity 
analysis.  Different  sensitivity  analyses  can  be  performed 
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to  systematically  test  the  sensitivity  of  the  total  utility 
scores  to  variations  in  the  weights  assigned  to  subfactors. 
Sensitivity  analysis  can  be  useful  to  a  group  of  intelli¬ 
gence  analysts,  for  example,  who  have  conflicting  opinions 
about  the  importance  of  particular  factors  in  the  hierarchy. 
The  analysis  would  reveal  whether  the  differences  in  opinion 
significantly  affect  the  resultant  total  utilities  of  the 
options.  By  showing  the  implications  of  differences  of 
opinion,  MAUA  decision  aids  like  ENCOA  reduce  the  emotional 
aspects  of  disagreement  by  promoting  a  task-focus  toward 
evaluating  which  differences  of  opinion  truly  make  a  differ¬ 
ence  overall. 

Sage  and  White  (1979)  have  recently  evaluated  a 
MAUA  decision  aid  called  RSCREEN  for  the  Command  and  Control 
Technical  Center,  Defense  Communications  Agency  (DCA/C140) . 
RSCREEN  is  extremely  similar  in  concept  to  ENCOA.  The  only 
difference  is  in  the  MAUA  hierarchy;  all  four  other  steps  in 
RSCREEN  are  identical  to  those  for  ENCOA.  The  evaluation 
results  strongly  supported  the  general  position  that  decision 
analytic  techniques  and  computerized  decision  aids  improve 
the  decision  process  in  general,  the  judgmental  accuracy 
of  individuals  in  particular,  and  the  communication  process 
of  the  decision-making  group.  According  to  Sage  and  White 
(1979,  p.  10); 

"RSCREEN  was  viewed  as  providing  a  realistic  and  ade¬ 
quate  situation  model  structure  which  encouraged  ac¬ 
curate  model  development  for  real-world  problems. 
Although  the  final  choice  selection  by  RSCREEN  was  not 
always  viewed  as  being  consistent  with  the  decision¬ 
maker's  original  choice,  there  were  indications  that 
the  aiding  process  could  be  convincing  enough  to 
change  a  decision-maker's  original  choice.  .  .  .  Deci¬ 
sion  process  changes  were  felt  by  those  using  the  aid 
to  be  such  as  to  lead  to  increased  and  more  effective 


thought  about  the  situation  of  interest,  particularly 
in  a  group  setting.  Improved  decision-making  capa¬ 
bility,  given  sufficient  training,  experience,  and 
awareness  of  RSCREEN's  situation  modeling  limitations, 
were  felt  to  be  likely  results  of  decision  process 
changes  due  to  using  the  aid  in  appropriate  circum¬ 
stances.  " 

According  to  Sage  and  White's  evaluation  of 
RSCREEN ,  the  major  barrier  to  the  effective  utilization  of 
computerized  decision  aids  is  the  personal  decision-making 
style  of  key  management  personnel  and  institutional  con¬ 
straints.  Unless  the  decision  aid  fits  into  both  the  man¬ 
agement  style  of  leadership  personnel  and  the  behavioral 
characteristics  of  the  operational  environment,  the  aid  will 
not  be  used.  Development  of  computerized  decision  aids  as 
an  adjunct  to  the  proposed  IPB  Templating  process  must, 
therefore,  explicitly  consider  potential  user  needs  and  or¬ 
ganizational  limitations.  By  so  doing,  analytically  correct 
aids  can  be  molded  to  the  analysts’  environment,  thus  in¬ 
creasing  the  probability  of  their  utilization.  The  inter¬ 
ested  reader  is  referred  to  Adelman,  Donnell,  Patterson,  and 
Weiss  (1980)  for  a  detailed  discussion  of  the  importance  of 
user  involvement  during  the  development  of  decision-analytic 
aids  to  their  successful  implementation. 

3 . 2  Research  and  Aids  Applicable  to  the  Revision  of  Deci¬ 
sion  Support  Templates 

The  iterative  templating  process  proposed  in  Circulars 
TC30-27  and  TC34-3  can  be  represented  quantitatively  by 
Bayes'  Theorem,  which  is  shown  in  equation  [1]. 
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The  situation  template  shows  how  the  enemy  force  would  probably 
look  for  different  courses  of  action;  therefore,  it  specifies, 
at  a  minimum,  the  initial  hypothesis  for  Bayes'  Theorem.  If 
the  situation  template  also  provides  an  initial  estimate  of 
the  most  likely  course  of  action,  as  we  assume  it  will  after 
analysts  have  developed  the  first  decision  support  templa-e, 
then  it  specifies  the  prior  probabilities  for  the  hypothesis 
in  some  qualitative  form. 

The  event  template  identifies  the  information  required 
to  support  each  of  the  initial  hypotheses.  Consequently, 
the  information  within  each  event  template  represents,  in 
qualitative  terms,  conditional  probabilities,  for  if  the 
enemy  actually  took  a  particular  course  of  attack,  then  cer¬ 
tain  events  have  a  higher  probability  of  being  observed  than 
others.  Furthermore,  the  information  within  all  the  event 
templates  represents,  again  in  qualitative  terms  at  present, 
likelihood  ratios  [i.e.,  P (D | ) /P (D | H2 ) ]  that  indicate  how 
much  more  likely  certain  events  are  to  be  observed  than 
others  for  one  course  of  attack  than  another. 

Finally,  the  decision  support  template  represents  the 
intelligence  estimate  in  graphic  form.  In  order  to  develop 
this  template,  the  analysts  must  use  collected  intelligence 
data  to  revise  their  initial  hypotheses  about  the  enemy's 
most  likely  course  of  attack.  Consequently,  the  decision 
support  template  represents,  again  in  qualitative  terms, 
the  posterior  probabilities  in  Bayes'  Theorem. 
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The  next  two  sections  provide  a  review  of  research 
studying  Bayesian  inference.  The  goal  throughout  this  re¬ 
view,  as  in  the  preceding  one,  is  to  identify  decision- 
analytic  techniques  and  computerized  decision  aids  that 
could  be  used  in  conjunction  with  templating  to  improve  the 
accuracy  of  the  necessary  judgments  in  the  proposed  IPB 
analysis  process. 

3.2.1  Research  findings;  without  judgment  aids  -  Ex¬ 
tensive  reviews  of  psychological  research  in  which  subjects' 
final  probability  estimates  have  been  compared  with  those 
prescribed  by  Bayes'  Theorem  can  be  found  in  Fischer,  Edwards 
and  Kelly  (1978) ,  Rapoport  and  Wallsten  (1972)  ,  and  Slovic 
and  Lichtenstein  (1971) .  "In  general,  humans  have  been 
found  to  be  very  suboptimal  processors  of  probabilistic  in¬ 
formation.  Although  they  typically  revise  their  opinions  in 
the  same  direction  as  Bayes'  Theorem,  they  do  not  revise 
them  enough"  (Fischer  et  al.,  p.  6). 

This  conclusion  is  based  on  a  variety  of  experi¬ 
mental  tasks  in  which  people  were  asked  to  infer  which  of 
two  or  more  statistical  models  (i.e.,  the  hypotheses)  gener¬ 
ated  the  data.  For  example,  Phillips  and  Edwards  (1966) 
used  binomial  data  generators,  Phillips,  Hays,  and  Edwards 
(1966)  used  multinominal  data  generators,  and  DuCharme  and 
Peterson  (1968)  used  normal  data  generators.  This  general 
result  of  suboptimality  in  judgment  has  been  called  conser¬ 
vatism  because  people  extract  less  certainty  from  the  data 
than  they  should,  and  consequently,  their  judgments  about 
the  implications  of  the  data  are  conservative  when  compared 
to  those  of  Bayes'  Theorem. 

This  finding  could  have  great  implications  for 
IPB,  which  is  essentially  a  sophisticated  Bayesian  inference 
task  as  we  noted  previously.  For  if  intelligence  analysts 
using  IPB  are  conservative  information  processors,  then  they 


are  not  drawing  implications  from  the  data  as  fast  as  they 
could  be  with  Bayes'  Theorem.  These  estimates  about  enemy 
courses  of  action  may  well  be  suboptimal  because  they  will 
not  have  sufficiently  revised  their  opinions  to  take  full 
account  of  the  certainty  in  the  data.  Consequently,  the 
entire  templating  process  will  not  convey  as  much  informa¬ 
tion  to  commanders  as  it  should.  The  time  available  for 
friendly  tactical  decision  planning  and  implementation  may 
be  reduced  considerably  if  intelligence  analysts  are  con¬ 
servative  information  processors. 

The  potential  implications  of  Bayesian  research 
for  IPB  are  compounded  by  the  fact  that  intelligence  ana¬ 
lysts  rely  on  language  to  convey  uncertain  information  in 
their  event  templates  and  event  analysis  matrices  instead  of 
numerical  estimates.  It  is  impossible,  however,  to  directly 
translate  qualitative  expressions  of  uncertainty  such  as 
"very  likely"  into  probability  values.  For  while  most 
people  would  agree  that  "very  likely"  means  a  probability 
greater  than  0.5,  there  is  no  general  agreement  of  how  much 
more  than  0.5  the  probability  is  (0.8?,  0.9?,  0.99?). 

The  lack  of  agreement  in  the  use  of  language  to 
convey  probability  estimates  has  been  demonstrated  with  in¬ 
telligence  analysts  in  a  number  of  experiments  reviewed  by 
Barclay,  Brown,  Kelly,  Peterson,  Phillips,  and  Selvidge 
(1977)  and  Phelps,  Halpin,  Johnson,  and  Moses  (1980)  .  The 
former,  for  example,  cites  an  anecdotal  study  where  an  in¬ 
telligence  analyst  was  asked  to  substitute  probability  esti¬ 
mates  for  some  of  the  verbal  qualifiers  in  an  article  he  had 
written.  The  first  statement  was:  "The  cease-fire  is  hold¬ 
ing  but  it  could  be  broken  within  a  week."  The  analyst  said 
that  he  meant  there  was  a  30%  chance  the  cease-fire  would  be 
broken  within  a  week.  Later,  an  analyst  who  had  helped  the 
original  analyst  prepare  the  statement  said  she  thought  that 
there  was  an  80%  chance  that  the  cease-fire  would  be  broken. 


Yet,  both  analysts  had  previously  believed  that  they  were  in 
agreement  about  what  could  happen. 

This  anecdote  and  more  importantly,  controlled 
research  with  analysts,  provides  strong  evidence  that  sig¬ 
nificant  miscommunication  occurs  among  intelligence  analysts. 
Additional  research  cited  by  Phelps  et  al.  (1980,  p.  10), 
"...indicates  that  a  simple  change  in  training  procedure,  a 
clarification  of  scale  definitions,  etc.,  would  not  be  ade¬ 
quate  to  significantly  improve  the  communication  of  the 
evaluation..."  Seeming  agreement  among  analysts  about  the 
implications  of  data  for  IPB  templates,  therefore,  might 
result  from  the  impreciseness  of  the  verbal  qualifiers  used 
in  the  IPB  process.  In  reality,  there  could  be  considerable 
disagreement.  When  such  disagreement  finally  surfaces,  the 
time  available  for  tactical  decision  planning  and  implemen¬ 
tation  again  may  be  lost  as  analysts  review  previously  de¬ 
veloped  templates  in  an  effort  to  determine  the  basis  of 
their  disagreement. 

The  fact  that  verbal  qualifiers  are  imprecise 
descriptions  of  levels  of  certainty  provides  strong  motiva¬ 
tion  for  the  use  of  a  quantitative  language  of  uncertainty. 
Appropriate  use  of  numbers  allows  uncertainty  to  be  ex¬ 
pressed  with  precision.  If  an  analyst  uses  numbers,  either 
percentages  or  odds,  to  convey  a  degree  of  belief  about  the 
likelihood  of  future  events,  and  the  numbers  are  carefully 
chosen  to  reflect  the  analyst's  uncertainty,  then  these 
numbers  can  be  easily  converted  into  probabilities. 

Research  is  beginning  to  identify  why  individ¬ 
uals  are  conservative  information  processors.  One  reason, 
for  example,  is  that  people  tend  to  search  for  confirming 
rather  than  disconf irming  evidence  of  alternative  hypotheses 
(see  Einhorn  and  Hogarth,  1978,  for  a  review).  This  is 
often  a  suboptimal  data  collection  and  revision  strategy 


because  data  often  can  confirm  many  hypotheses  at  the  same 
time;  disconf irming  evidence,  on  the  other  hand,  can  quickly 
eliminate  a  hypothesis  from  consideration.  Another  reason 
why  individuals  are  conservative  information  processors  is 
that  they  tend  to  use  simple  rules  or  "judgment  heuristics" 
(Tversky  and  Kahneman,  1974)  instead  of  the  axioms  of  proba¬ 
bility  theory  when  making  subjective  probability  estimates. 

As  a  result  of  learning  why  individuals  are  conservative 
information  processors,  researchers  have  been  able  to  de¬ 
velop  decision  aids  and  training  methods  to  help  persons 
make  more  accurate  probability  estimates. 

A  recent  review  paper  by  Phelps  et  al.  (1980) 
suggests  that  training  methods  can  be  developed  to  improve 
the  accuracy  of  probability  estimates.  They  cited  two  ex¬ 
periments  by  Lichtenstein  and  Fischhoff  (1978)  who  found 
that  people  were  able  to  make  more  accurate  probability 
judgments  after  recieving  feedback  about  the  direction  and 
magnitude  of  their  initial  errors.  However,  they  found  that 
the  transfer  of  this  learning  to  other  tasks  was  moderate, 
at  best.  Donnell  and  DuCharme  (1975)  have  found  similar 
results.  As  a  result,  Phelps  et  al.  (1980)  suggested  that 
intelligence  analysts'  training  in  probability  assessment  be 
confined  to  the  tactical  intelligence  context.  This  can  be 
readily  accomplished,  since  the  training  methods  used  by 
Lichtenstein  and  Fischhoff  (1978),  "...are  well  documented 
and  could  easily  be  automated  for  self-instruction  and  prac¬ 
tice"  (Phelps  et  al.,  1980,  p.  20). 

In  addition  to  training  methods,  analytical  de¬ 
cision  aids  have  been  developed  to  help  people  make  more  ac¬ 
curate  probability  estimates.  In  the  next  section  we  discuss 
the  research  supporting  their  development;  in  a  subsequent 
section  we  discuss  how  they  might  be  used  to  improve  IPB. 


3.2.2  Research  finding;  with  judgment  aids  -  Efforts 
to  develop  judgment  aids  to  help  people  infer  posterior 
probabilities  began  in  the  1960's  with  the  development  of  an 
inference  system  called  PIP — an  acronym  for  Probabilistic 
Information  Processing.  In  the  formulation  proposed  by 
Edwards,  Lindman,  and  Phillips  (1965),  people  were  tasked 
with  identifying  relevant  hypotheses,  information  sources 
that  could  discriminate  between  these  hypotheses,  and  the 
likelihood  ratios  linking  data  with  hypotheses.  This  would 
be  analogous  to  developing,  in  more  quantitative  terms  than 
present,  the  situation  and  event  templates  in  IPB.  The  task 
of  aggregating  information  across  data  in  PIP  was  assigned 
to  Bayes'  Theorem,  since  research  had  indicated  that  people 
were  conservative  information  possessors.  This  would  be 
analogous  to  using  Bayes'  Theorem  to  integrate  the  intel¬ 
ligence  data  used  to  make  the  intelligence  estimate  repre¬ 
sented  graphically  in  the  decision-support  template. 

Initial  efforts  by  Edwards,  Phillips,  Hays,  and 
Goodman  (1968),  Kaplan  and  Newman  (1966),  and  Wheeler  (1972) 
all  found  PIP  superior  to  unaided  inference;  PIP  consis¬ 
tently  assigned  higher  posterior  probabilities  to  the  true 
hypotheses.  Implicit  in  the  original  formulation  of  PIP  was 
the  assumption  that  the  environment  could  be  described  in 
terms  of  a  stationary,  single-stage  inference  model  in  which 
all  data  were  conditionally  independent  of  the  hypotheses. 
Subsequent  efforts  to  test  PIP  varied  aspects  of  this  origi¬ 
nal  assumption,  thereby  representing  more  complex,  yet  rep¬ 
resentative  environments. 

Research  in  more  complex  environments  also  sup¬ 
ported  the  value  of  PIP,  but  the  results  were  less  clear. 

For  example,  Domas  and  Peterson  (1972)  found  PIP  to  outper¬ 
form  unaided  judgment  when  data  were  conditionally  indepen¬ 
dent,  but  not  when  the  data  were  conditionally  dependent. 
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for  in  the  latter  case  PIP  assigned  excessively  high  proba¬ 
bilities.  In  contrast,  Schum,  Southard,  and  Wombolt  (1969) 
found  a  modified  PIP,  termed  semi-PIP,  to  perform  quite  well 
with  conditionally  dependent  data.  In  semi-PIP,  persons 
first  grouped  data  into  bundles  that  were  conditionally 
independent,  although  the  data  within  each  bundle  was  con¬ 
ditionally  dependent.  Persons  then  assessed  the  likelihood 
ratios  for  the  conditionally  independent  bundles,  and  Bayes 
Theorem  combined  these  estimates  into  posterior  probabili¬ 
ties.  Semi-PIP  was  found  to  be  substantially  better  than 
unaided  judgments,  particularly  as  the  number  of  data  to  be 
aggregated  increased. 

PIP  Systems  have  also  been  developed  to  deal 
with  the  problem  of  hierarchical  (also  called  multistage  or 
cascaded)  inference.  Hierarchical  inference  problems  in¬ 
volve  several  levels  of  analysis  and  therefore,  are  repre¬ 
sentative  of  many  intelligence  problems.  Experimental 
studies  of  intuitive  hierarchical  inference  by  Gettys, 

Kelly,  and  Peterson  (1973)  and  Schum,  DuCharme,  and  DePitts 
(1971)  have  also  found  humans  to  be  suboptimal  information 
processors.  The  reason  for  this,  however,  is  that  their 
posterior  probabilities  were  too  extreme  when  compared  to 
those  of  Bayes'  Theorem.  To  the  extent  that  IPB  incorpor¬ 
ates  hierarchical  inference,  it  is  susceptible  to  yet  an¬ 
other  limitation  in  unaided  judgment. 

Kelly  and  Barclay  (1973)  provide  general  mathe¬ 
matical  models  for  hierarchical  inference  that  can  be  di¬ 
rectly  translated  into  computer  algorithms  for  decision 
aids.  And  Barclay  (1976)  describes  an  interactive  graphics 
aid  for  Bayesian  hierarchical  inference.  From  a  practical 
standpoint,  however,  these  models  are  tractable  only  when 
the  problem  can  be  structured  to  eliminate  most  conditional 


69 


dependencies  between  data.  If  this  cannot  be  done,  indi¬ 
viduals  would  be  required  to  make  a  prohibitively  large 
number  of  judgments. 

Experimental  evidence  on  hierarchical  PIP  sys¬ 
tems  is  unfortunately  scant.  Gettys,  Kelly,  Peterson, 
Michel,  and  Steiger  (1973)  have  conducted  two  relevant 
studies,  however,  both  of  which  demonstrated  the  superiority 
of  a  hierarchical  PIP  system  over  unaided  inference. 

Hierarchical  PIP  systems  (called  Bayesian  hier¬ 
archical  inference  models)  have  been  developed  for  a  number 
of  strategic  intelligence  problems.  For  example,  Barclay, 
Kelly,  and  Stewart  (1976)  developed  a  hierarchical  PIP  sys¬ 
tem  for  assessing  the  personnel  strength  of  foreign  ground 
force  units.  And  Peterson,  Randall,  Shawcross,  and  Ulvila 
(1976)  developed  a  hierarchical  PIP  system  for  the  Navy  to 
predict  an  anti-ship  missile  threat.  Since  the  PIP  system 
developed  by  Barclay  et  al.  (1976)  is  classified,  we  will 
discuss  the  PIP  system  developed  by  Peterson  et  al.  (1976) 
to  better  help  the  reader  understand  its  conceptual  frame¬ 
work. 

The  structure  of  the  Bayesian  hierarchical  model 
developed  for  a  Red  air/submarine  threat  is  presented  in 
Figure  3-4.  In  this  model,  a  series  of  different  enemy 
activities,  and  the  associated  observations  that  might  be 
available  to  a  friendly  task  force  commander,  were  postu¬ 
lated.  Then  conditional  probabilities  were  assessed  linking 
the  activities  to  the  hypotheses  of  routine  surveillance, 
feint  attack,  and  attack,  and  linking  the  observations  to 
the  activities.  For  example,  consider  the  targeting  ac¬ 
tivity.  It  was  hypothesized  that  Red's  targeting  activity 
might  occur  in  one  of  three  ways:  intermittently,  steadily, 
or  not  at  all.  If  Red  intended  no  attack,  thus  supporting 
the  "routine"  hypothesis,  it  was  most  likely  that  Red  would 
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not  engage  in  any  targeting.  Consequently,  conditional 
probabilities  of  .10,  .05,  and  .85  were  assessed  for  the 
activities  of  intermittent,  steady,  and  none,  respectively, 
given  the  "routine"  hypothesis,  as  shown  in  the  top  matrix 
in  Figure  3-4.  If  Red  intended  feint,  it  was  thought  that, 
most  likely,  Red  would  target  in  a  steady  fashion.  Conse¬ 
quently,  conditional  probabilities  of  .27,  .68,  and  .05  were 
assessed  for  intermittent,  steady,  and  none,  respectively. 
Finally,  if  Red  intended  to  attack,  it  was  most  likely  that 
Red  would  try  to  conceal  the  target  signal  by  using  it  in¬ 
termittently;  consequently,  conditional  probabilities  of 
.71,  .28,  and  .01  were  assessed  for  intermittent,  steady, 
and  none. 

In  a  similar  manner,  the  bottom  matrix  of  Figure 
3-4,  linking  observations  to  activities,  was  assessed.  If 
Red  actually  was  targeting  intermittently  (an  activity) ,  it 
was  most  likely  that  the  task  force  commander  would  receive 
a  report  that  Red  was  not  targeting  (an  observation).  Thus, 
conditional  probabilities  of  .30,  0,  and  .70  were  assessed 
for  the  observation  of  intermittent,  steady,  and  none,  re¬ 
spectively.  Further,  conditional  probabilities  of  .10,  .60, 
and  .30  were  assessed  for  the  observation  of  intermittent, 
steady,  and  none  given  that  Red's  activity  was  steady  tar¬ 
geting  and  probabilities  of  .04,  .01,  and  .95  for  observa¬ 
tions  of  intermittent,  steady,  and  none  if  Red  was  not 
targeting. 

All  the  conditional  probabilities  in  the  hier¬ 
archical  model  shown  in  Figure  3-4  had  to  be  assessed  by 
people.  Once  the  conditional  probabilities  have  been  en¬ 
tered,  however,  they  are  combined  analytically  (arithmeti¬ 
cally)  ,  not  intuitively,  in  the  PIP  system.  Specifically, 
by  multiplying  the  lower  matrix  of  Figure  3-4,  which  links 
the  observations  to  the  activities,  by  the  upper  matrix, 
which  links  the  activities  to  the  hypotheses,  yields  a 


product  matrix  that  relates  the  observations  to  the  hypothe¬ 
ses.  Table  3-1  shows  this  multiplication  for  the  targeting 
example  explained  above.  This  product  matrix  provides  out¬ 
put  in  the  desired  form  of  an  indicator  list  relating  obser¬ 
vations  to  hypotheses,  in  terms  of  the  probability  that  the 
former  implies  the  latter.  As  a  result,  the  overall  proba¬ 
bility  of  each  hypothesis  (i.e.,  possible  enemy  intent)  now 
can  be  determined  explicitly  on  the  basis  of  the  observa¬ 
tions  made  at  a  particular  time. 

As  this  example  illustrates,  hierarchical  PIP 
systems  have  been  developed  to  help  people  make  complex 
probability  judgments.  We  now  turn  to  consider  directly  the 
applicability  of  general  PIP  decision  aids  in  IPB. 

3.2.3  A  judgment  aid  -  It  is  important  to  reiterate  at 
the  outset  that  the  judgment  process  supporting  the  revision 
of  templates  within  IPB  can  be  conceptualized  in  terns  of 
the  elements  of  Bayes '  Theorem.  The  decision  support  tem¬ 
plate  at  the  end  of  one  iteration  is  the  situation  template 
at  the  beginning  of  the  next  iteration.  The  situation  tem¬ 
plate  is  represented  by  the  prior  probabilities,  which  indi¬ 
cate  the  relative  likelihood  of  the  different  OPFOR  courses 
of  action  (i.e.,  hypotheses)  under  consideration.  The  event 
templates  and  decision  matrices  are  represented  by  the  con¬ 
ditional  probabilities,  which  indicate  the  relative  likeli¬ 
hood  that  certain  events  support  particular  courses  of 
action.  The  posterior  probabilities  are  represented  by  the 
decision  support  template,  which  indicates  the  revised  like¬ 
lihood  of  the  courses  of  action  (i.e.,  hypotheses)  on  the 
basis  of  observed  data.  This  new  estimate  of  enemy  intent 
is  then  input  to  friendly  tactical  decision  making  and  sub¬ 
sequent  action. 

The  literature  cited  previously  indicates  that 
people  have  considerable  difficulty  in  making  judgments 
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consistent  with  Bayes'  Theorem.  People  extract  less  cer¬ 
tainty  from  the  data  than  they  should,  and  consequently, 
their  judgments  about  the  implications  of  the  data  are  con¬ 
servative  when  compared  to  those  of  Bayes'  Theorem.  Since 
the  judgment  process  supporting  the  revision  of  IPB  tem¬ 
plates  can  be  conceptualized  as  a  Bayesian  inference  pro¬ 
cess,  this  finding  could  have  great  implications  for  IPB. 

For  if  intelligence  analysts  using  IPB  are  conservative 
information  processors,  then  they  are  not  drawing  implica¬ 
tions  from  the  data  as  fast  as  they  could  be  with  Bayes' 
Theorem.  Their  estimates  about  enemy  courses  of  action  may 
well  be  suboptimal  because  they  will  not  have  sufficiently 
revised  their  opinions  to  take  full  account  of  the  certainty 
in  the  data.  Consequently,  the  entire  templating  process 
will  not  convey  as  much  information  to  commanders  as  it 
should.  As  a  result,  force  commanders  will  lose  time  for 
tactical  decision  making. 

General  decision  aids,  called  Probabilistic  In¬ 
formation  Processing  (PIP)  systems  have  been  developed  to 
ensure  that  human  judgment  is  consistent  with  Bayes'  Theorem 
for  a  number  of  intelligence  problems  (e.g.,  see  Peterson 
et  al.,  1976).  Such  a  decision  aid  is  now  being  developed 
to  help  tactical  intelligence  analysts  revise  their  judgment 
about  the  most  likely  OPFOR  course  of  action  on  the  basis  of 
new  information.  This  aid  is  being  designed  so  that  it  can 
be  readily  integrated  into  the  proposed  iterative  IPB 
analysis  process.  Consequently,  it  will  provide  an  impor¬ 
tant  adjunct  to  the  development  and  utilization  of  IPB 
templates.  The  decision  aid  is  scheduled  for  initial  evalu¬ 
ation  during  1981. 

The  Bayesian  judgment  aid  will  probably  operate 
in  the  following  manner: 


(1)  the  analysts  define  the  n  different  enemy 
courses  of  action  (COAs)  under  consideration  (i.e.,  enter  a 
brief  l<  10  character  title] ) ; 

(2)  the  analysts  enter  a  set  of  prior  proba¬ 
bilities  for  the  n  potential  courses  of  action; 

(3)  for  a  given  datum,  the  analysts  input  a 
brief  title  and  the  probability  of  that  datum  conditional 
upon  each  COA  being  considered; 

(4)  the  analysts  inspect  the  posterior  proba¬ 
bilities  (or  likelihood  ratios) ; 

(5)  they  revise  any  posterior  probabilities 
that  are  counter-intuitive;  and 

(6)  the  analysts  report  on  potential  enemy  COAs 
based  or  the  probabilities  after  Step  5  or  return  to  Step  3 
if  there  are  additional  data. 

Step  5  is  necessary  since  any  redundancy  or 
facilitation  in  the  data  is  not  likely  to  be  taken  into 
account  in  providing  the  probability  estimates  in  Step  3. 

For  the  probabilities  in  Step  4  to  always  be  correct,  with¬ 
out  some  revision  at  Step  5,  all  data  must  be  independent. 

An  alternative  to  Step  5  would  be  to  require  that  the  proba¬ 
bilities  estimated  in  Step  3  be  conditional  not  only  on  the 
hypothesized  COA  but  also  upon  all  data  that  have  come 
before.  This  type  of  probability  estimate  is  quite  diffi¬ 
cult,  however,  and  is  most  easily  performed  using  a  Bayesian 
hierarchical  (staged)  inference  (BHI)  model  rather  than  the 
simple  model  proposed  for  use  here. 
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The  ability  to  use  a  Bayesian  framework  to 
represent  the  different  judgments  inherent  in  revising  the 
IPB  templates  is  illustrated  in  the  above  steps.  Step  1 
represents  the  different  OPFOR  courses  of  action  represented 
in  the  situation  template.  Step  2  represents  the  relative 
likelihood  of  these  actions  at  the  end  of  one  iteration  of 
the  IPB  analysis  process.  Step  3  represents  the  judgments 
in  the  event  templates  and  event  matrices,  which  indicate 
the  relative  likelihood  of  having  collected  the  newly  acquired 
data  on  the  basis  of  the  enemy  actually  taking  different 
courses  of  action.  Steps  4,  5,  and  6  represent  the  intel¬ 
ligence  analysts'  estimates  of  enemy  intent,  as  represented 
graphically  in  the  decision-support  template. 

It  is  important  to  note  that  a  Bayesian  aid  also 
can  be  used  even  if  there  is  no  revision  of  IPB  templates. 

In  this  case,  the  prior  probabilities  are  assumed  to  be 
equal  for  each  enemy  course  of  action  unless  the  analysts 
think  certain  actions  are  more  likely  than  others  (Step  2). 

The  analysts  then  either  estimate  (1)  the  probability  of  the 
individual  intelligence  datum  conditional  on  each  of  the 
hypothesized  courses  of  action  as  the  datum  is  collected 
(Step  3),  or  (2)  the  probability  of  all  collected  intelli¬ 
gence  data  conditional  on  each  action.  In  the  former  case, 
Bayes’  Theorem  generates  posterior  probabilities  after 
collecting  each  datum;  the  posterior  probabilities  are  then 
the  prior  probabilities  that  are  revised  on  the  basis  of  the 
next  datum.  In  the  latter  case,  Bayes’  Theorem  generates 
the  final  posterior  probabilities.  The  former  case  is 
recommended  in  order  to  minimize  conservatism  in  the  ana¬ 
lysts’  final  estimate  of  enemy  intent.  Notice  that  while 
the  posterior  probabilities  are  constantly  revised  on  the 
basis  of  new  information,  the  decision-support  template  is 
not  because  it  is  not  developed  until  the  analysts  have 
collected  the  data  necessary  for  the  final  estimate  of  enemy 
intent. 


ENCOA  also  can  be  used  for  revising  decision 
support  templates  on  the  basis  of  collected  intelligence 
data.  During  the  next  iteration,  the  analysts  modify  (1) 
the  score  of  each  OPFOR  course  of  action  on  each  of  the 
attributes  in  the  hierarchy  to  represent  the  implications  of 
collected  data,  and  (2)  the  weights  on  the  attributes  at 
each  level,  if  the  collected  data  has  altered  the  relative 
importance  of  going  from  the  lowest  to  the  highest  scores  on 
the  attributes.  The  scores  and  weights  are  then  combined  to 
indicate  the  most  likely  enemy  course  of  action.  Again, 
sensitivity  analysis  provides  a  means  for  determining  the 
effect  of  using  different  scores  and  weights  on  overall 
judgments  of  enemy  intent.  The  overall  utility  scores  for 
each  OPFOR  course  of  action  can  be  readily  represented  in 
graphical  form  in  the  next  decision-support  template. 

An  advantage  of  using  Bayes'  Theorem  instead  of 
multi-attribute  utility  theory  to  represent  quantitatively 
the  judgments  inherent  in  template  revision  is  that  the  pos¬ 
terior  probabilities  can  be  readily  incorporated  into  an 
expected  utility  theoretic  framework  for  friendly  tactical 
decision  making.  This  framework  is  represented  pictorially 
in  the  decision  tree  shown  in  Figure  3-5.  in  this  represen¬ 
tation,  the  friendly  commander  can  take  either  action  #1  or 
action  #2.  Each  action  will  have  a  particular  outcome  (or 
consequence) ,  depending  on  the  course  of  action  actually 
adopted  by  the  enemy.  Furthermore,  each  outcome  will  vary 
in  utility  (or  value)  to  the  friendly  commander.  Actual 
enemy  intent  is,  of  course,  assumed  to  be  unknown  at  the 
time  of  the  decision. 

Bayes'  Theorem  can  be  used  to  indicate  the 
probability  (or  likelihood)  that  the  enemy  is  taking  course 
of  action  A  or  B  on  the  basis  of  collected  data.  This 
information  can  be  represented  pictorially  in  the  decision- 
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support  template  presented  to  the  friendly  commander.  In 
addition,  Bayes'  Theorem  can  be  used  to  analytically  revise 
the  relative  likelihood  estimates  for  different  enemy  courses 
of  action  (e.g.,  A  and  B  in  Figure  3-3)  on  the  basis  of 
newly  collected  data  organized  in  the  decision  matrices 
proposed  in  Circulars  TC30-27  and  TC34-3.  Again,  the  rela¬ 
tive  likelihood  estimates  can  be  presented  to  the  friendly 
commander  in  the  decision-support  template. 

Expected  utility  theory  provides  an  analytic 
procedure  for  combining  these  relative  likelihood  estimates 
with  the  friendly  commanders'  estimates  of  the  utility  of 
different  outcomes  resulting  from  their  and  the  enemy's 
actions.  In  terms  of  Figure  3-3,  the  friendly  commander 
should,  first,  multiply  the  utility  and  probability  estimate 
for  each  of  the  four  branches  of  the  decision  tree  shown  in 
Figure  3-3,  and  then,  sum  the  two  resulting  values  for  each 
action  to  determine  the  expected  utility  of  each  action. 

The  commander  then  should  select  the  action  with  the  highest 
expected  utility  in  order  to  ensure  the  greatest  degree  of 
success  over  time.  In  this  fashion,  the  posterior  probability 
judgments  of  enemy  intent  that  are  inherent  in  template 
revision  can  be  readily  incorporated  into  friendly  tactical 
decision  making  through  an  explicit,  retraceable  analysis 
process. 

3. 3  Summary 

IPB  depends  on  the  subjective  judgment  of  intelligence 
analysts.  These  judgments  can  be  represented  conceputally 
in  terms  of  a  causal  inference  model,  for  analysts  must 
infer  enemy  intent  on  the  basis  of  tactical  indicators 
and  information.  Research  on  causal  inference  with  experts 
in  different  fields  indicates  that  the  level  of  judgmental 
accuracy  and  interpersonal  agreement  is  a  direct  function 
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of  the  characteristics  of  the  judgmental  task  facing  the 
experts.  This  conclusion  is  further  supported  by  controlled 
laboratory  research,  which  has  shown  (1)  that  judgmental 
accuracy  and  agreement  can  be  increased  or  decreased  by 
manipulating  characteristics  of  the  task,  and  (2)  that 
relatively  high  levels  of  accuracy  and  agreement  can  be 
maintained  under  conditions  that  normally  prevent  it,  by 
using  judgment  aids  that  provide  persons  with  information 
about  task  characteristics. 

This  section  describes  two  judgment  aids  for  improving 
the  judgmental  process  inherent  in  IPB.  The  first  aid  uses 
multi-attribute  utility  assessment  techniques  to  facilitate 
the  development  of  the  initial  decision-support  template. 

The  judgment  aid  is  called  ENCOA,  for  Enemy  Courses  of 
Action.  ENCOA  provides  analysts  with  a  systematic  procedure 
for  evaluating  each  potential  OPFOR  course  of  action  on 
twenty-four  factors  affecting  enemy  intent.  In  addition, 

ENCOA  can  be  used  to  present  quantitatively  and  pictorially 
the  scores,  weights,  overall  utilities,  and  subsequent 
sensitivity  analyses  to  the  friendly  commander.  Such  infor¬ 
mation  describing  (1)  how  the  analysts  reached  their  con¬ 
clusions  about  the  most  likely  OPFOR  course  of  action,  as 
well  as  (2)  the  implications  of  differences  in  opinion 
between  the  analysts,  represent  an  important  adjunct  to  the 
graphic  decision-support  template. 

The  second  decision  aid  uses  Bayesian  assessment  tech¬ 
niques  to  facilitate  the  revision  of  decision-support 
templates  on  the  basis  of  newly  collected  data  represented 
in  the  event  templates  and  event  analysis  matrices.  The 
iterative  process  for  revising  IPB  templates  can  be  represented 
quantitatively  by  Bayes'  Theorem.  Psychological  research 
comparing  subjects'  final  probability  estimates  with  those 
prescribed  by  Bayes'  Theorem  has  shown  that  subjects  do  not 


revise  their  final  probability  estimates  far  enough;  they 
are  conservative.  Therefore,  Bayesian  judgment  aids,  called 
Probabilistic  Information  Processing  systems,  should  help 
analysts  draw  better  implications  from  collected  intelligence 
data.  These  implications,  represented  as  probability  estimates, 
can  then  be  incorporated  into  the  event  and  decision  support 
templates.  Consequently,  such  aids  should  help  analysts 
implement  the  template  revision  process  outlined  in  Circulars 
TC30-27  and  TC34-3. 


4.0  CONCLUSION 


Intelligence  Preparation  of  the  Battlefield  represents 
a  major  step  toward  improving  procedures  for  correlating 
data  with  enemy  intent.  However,  it  can  be  improved,  for 
the  process  whereby  input  (new  intelligence  data)  is  trans¬ 
lated  into  output  (templates)  necessarily  depends  on  ana¬ 
lysts'  judgments  about  the  implications  of  multiple  pieces 
of  potentially  fallible  data.  Except  for  terrain  analysis 
(Step  3)  and  weather  analysis  (Step  4),  Circulars  TC30-27 
and  TC34-3  fail  to  tell  intelligence  analysts  how  they  are 
to  make  the  judgments  necessary  to  implement  the  proposed 
IPB  analysis  process.  Furthermore,  they  provide  little  in¬ 
formation  on  how  analysts  are  to  revise  event  and  decision- 
support  templates  on  the  basis  of  new  information.  Finally, 
they  fail  to  describe  decision-analytic  techniques  and 
judgmental  aids  that  are  available  (or  being  developed)  to 
help  analysts  make  the  necessary  judgments.  Scientific 
research  on  causal  inference  strongly  suggests  that  such 
techniques  and  aids  would  improve  (1)  the  cognitive  skills 
and,  thus,  the  judgmental  accuracy  of  intelligence  analysts 
developing  IPB  templates,  and  (2)  the  communication  process 
for  conveying  the  reasons  for  these  judgments  to  tactical 
commanders. 
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